mirror of
https://github.com/donnemartin/data-science-ipython-notebooks.git
synced 2024-03-22 13:30:56 +08:00
Merge pull request #20 from donnemartin/develop
Add TensorFlow tutorials notebooks.
This commit is contained in:
Donne Martin
commit
e5680f2ca0
60
README.md
60
README.md
|
|
@ -85,20 +85,58 @@ IPython Notebook(s) used in [kaggle](https://www.kaggle.com/) competitions and b
|
|||
|
||||
IPython Notebook(s) demonstrating deep learning functionality.
|
||||
|
||||
<br/>
|
||||
<p align="center">
|
||||
<img src="https://avatars0.githubusercontent.com/u/15658638?v=3&s=100">
|
||||
</p>
|
||||
|
||||
### tensor-flow-tutorials
|
||||
|
||||
| Notebook | Description |
|
||||
|--------------------------------------------------------------------------------------------------------------|-------------------------------------------------------------------------------------------------------------------------------------------------------------------|
|
||||
| [tsf-basics](http://nbviewer.ipython.org/github/donnemartin/data-science-ipython-notebooks/blob/master/deep-learning/tensor-flow-examples/1_intro/basic_operations.ipynb) | Learn basic operations in TensorFlow, a library for various kinds of perceptual and language understanding tasks from Google. |
|
||||
| [tsf-linear](http://nbviewer.ipython.org/github/donnemartin/data-science-ipython-notebooks/blob/master/deep-learning/tensor-flow-examples/2_basic_classifiers/linear_regression.ipynb) | Implement linear regression in TensorFlow. |
|
||||
| [tsf-logistic](http://nbviewer.ipython.org/github/donnemartin/data-science-ipython-notebooks/blob/master/deep-learning/tensor-flow-examples/2_basic_classifiers/logistic_regression.ipynb) | Implement logistic regression in TensorFlow. |
|
||||
| [tsf-nn](http://nbviewer.ipython.org/github/donnemartin/data-science-ipython-notebooks/blob/master/deep-learning/tensor-flow-examples/2_basic_classifiers/nearest_neighbor.ipynb) | Implement nearest neighboars in TensorFlow. |
|
||||
| [tsf-alex](http://nbviewer.ipython.org/github/donnemartin/data-science-ipython-notebooks/blob/master/deep-learning/tensor-flow-examples/3_neural_networks/alexnet.ipynb) | Implement AlexNet in TensorFlow. |
|
||||
| [tsf-cnn](http://nbviewer.ipython.org/github/donnemartin/data-science-ipython-notebooks/blob/master/deep-learning/tensor-flow-examples/3_neural_networks/convolutional_network.ipynb) | Implement convolutional neural networks in TensorFlow. |
|
||||
| [tsf-mlp](http://nbviewer.ipython.org/github/donnemartin/data-science-ipython-notebooks/blob/master/deep-learning/tensor-flow-examples/3_neural_networks/multilayer_perceptron.ipynb) | Implement multilayer perceptrons in TensorFlow. |
|
||||
| [tsf-rnn](http://nbviewer.ipython.org/github/donnemartin/data-science-ipython-notebooks/blob/master/deep-learning/tensor-flow-examples/3_neural_networks/recurrent_network.ipynb) | Implement recurrent neural networks in TensorFlow. |
|
||||
| [tsf-gpu](http://nbviewer.ipython.org/github/donnemartin/data-science-ipython-notebooks/blob/master/deep-learning/tensor-flow-examples/4_multi_gpu/multigpu_basics.ipynb) | Learn about basic multi-GPU computation in TensorFlow. |
|
||||
| [tsf-gviz](http://nbviewer.ipython.org/github/donnemartin/data-science-ipython-notebooks/blob/master/deep-learning/tensor-flow-examples/5_ui/graph_visualization.ipynb) | Learn about graph visualization in TensorFlow. |
|
||||
| [tsf-lviz](http://nbviewer.ipython.org/github/donnemartin/data-science-ipython-notebooks/blob/master/deep-learning/tensor-flow-examples/5_ui/loss_visualization.ipynb) | Learn about loss visualization in TensorFlow. |
|
||||
|
||||
### tensor-flow-exercises
|
||||
|
||||
| Notebook | Description |
|
||||
|--------------------------------------------------------------------------------------------------------------|-------------------------------------------------------------------------------------------------------------------------------------------------------------------|
|
||||
| [tsf-not-mnist](http://nbviewer.ipython.org/github/donnemartin/data-science-ipython-notebooks/blob/master/deep-learning/tensor-flow-exercises/1_notmnist.ipynb) | Learn simple data curation by creating a pickle with formatted datasets for training, development and testing in TensorFlow. |
|
||||
| [tsf-fully-connected](http://nbviewer.ipython.org/github/donnemartin/data-science-ipython-notebooks/blob/master/deep-learning/tensor-flow-exercises/2_fullyconnected.ipynb) | Progressively train deeper and more accurate models using logistic regression and neural networks in TensorFlow. |
|
||||
| [tsf-regularization](http://nbviewer.ipython.org/github/donnemartin/data-science-ipython-notebooks/blob/master/deep-learning/tensor-flow-exercises/3_regularization.ipynb) | Explore regularization techniques by training fully connected networks to classify notMNIST characters in TensorFlow. |
|
||||
| [tsf-convolutions](http://nbviewer.ipython.org/github/donnemartin/data-science-ipython-notebooks/blob/master/deep-learning/tensor-flow-exercises/4_convolutions.ipynb) | Create convolutional neural networks in TensorFlow. |
|
||||
| [tsf-word2vec](http://nbviewer.ipython.org/github/donnemartin/data-science-ipython-notebooks/blob/master/deep-learning/tensor-flow-exercises/5_word2vec.ipynb) | Train a skip-gram model over Text8 data in TensorFlow. |
|
||||
| [tsf-lstm](http://nbviewer.ipython.org/github/donnemartin/data-science-ipython-notebooks/blob/master/deep-learning/tensor-flow-exercises/6_lstm.ipynb) | Train a LSTM character model over Text8 data in TensorFlow. |
|
||||
| [theano-intro](http://nbviewer.ipython.org/github/donnemartin/data-science-ipython-notebooks/blob/master/deep-learning/theano-tutorial/intro_theano/intro_theano.ipynb) | Intro to Theano, which allows you to define, optimize, and evaluate mathematical expressions involving multi-dimensional arrays efficiently. It can use GPUs and perform efficient symbolic differentiation. |
|
||||
| [theano-scan](http://nbviewer.ipython.org/github/donnemartin/data-science-ipython-notebooks/blob/master/deep-learning/theano-tutorial/scan_tutorial/scan_tutorial.ipynb) | Learn scans, a mechanism to perform loops in a Theano graph. |
|
||||
| [theano-logistic](http://nbviewer.ipython.org/github/donnemartin/data-science-ipython-notebooks/blob/master/deep-learning/theano-tutorial/intro_theano/logistic_regression.ipynb) | Implement logistic regression in Theano. |
|
||||
| [theano-rnn](http://nbviewer.ipython.org/github/donnemartin/data-science-ipython-notebooks/blob/master/deep-learning/theano-tutorial/rnn_tutorial/simple_rnn.ipynb) | Implement recurrent neural networks in Theano. |
|
||||
| [theano-mlp](http://nbviewer.ipython.org/github/donnemartin/data-science-ipython-notebooks/blob/master/deep-learning/theano-tutorial/theano_mlp/theano_mlp.ipynb) | Implement multilayer perceptrons in Theano. |
|
||||
| [deep-dream](http://nbviewer.ipython.org/github/donnemartin/data-science-ipython-notebooks/blob/master/deep-learning/deep-dream/dream.ipynb) | Caffe-based computer vision program which uses a convolutional neural network to find and enhance patterns in images. |
|
||||
| [tsf-fully-connected](http://nbviewer.ipython.org/github/donnemartin/data-science-ipython-notebooks/blob/master/deep-learning/tensor-flow-exercises/2_fullyconnected.ipynb) | Progressively train deeper and more accurate models using logistic regression and neural networks in TensorFlow. |
|
||||
| [tsf-regularization](http://nbviewer.ipython.org/github/donnemartin/data-science-ipython-notebooks/blob/master/deep-learning/tensor-flow-exercises/3_regularization.ipynb) | Explore regularization techniques by training fully connected networks to classify notMNIST characters in TensorFlow. |
|
||||
| [tsf-convolutions](http://nbviewer.ipython.org/github/donnemartin/data-science-ipython-notebooks/blob/master/deep-learning/tensor-flow-exercises/4_convolutions.ipynb) | Create convolutional neural networks in TensorFlow. |
|
||||
| [tsf-word2vec](http://nbviewer.ipython.org/github/donnemartin/data-science-ipython-notebooks/blob/master/deep-learning/tensor-flow-exercises/5_word2vec.ipynb) | Train a skip-gram model over Text8 data in TensorFlow. |
|
||||
| [tsf-lstm](http://nbviewer.ipython.org/github/donnemartin/data-science-ipython-notebooks/blob/master/deep-learning/tensor-flow-exercises/6_lstm.ipynb) | Train a LSTM character model over Text8 data in TensorFlow. |
|
||||
|
||||
<br/>
|
||||
<p align="center">
|
||||
<img src="http://www.deeplearning.net/software/theano/_static/theano_logo_allblue_200x46.png">
|
||||
</p>
|
||||
|
||||
### theano-tutorials
|
||||
|
||||
| Notebook | Description |
|
||||
|--------------------------------------------------------------------------------------------------------------|-------------------------------------------------------------------------------------------------------------------------------------------------------------------|
|
||||
| [theano-intro](http://nbviewer.ipython.org/github/donnemartin/data-science-ipython-notebooks/blob/master/deep-learning/theano-tutorial/intro_theano/intro_theano.ipynb) | Intro to Theano, which allows you to define, optimize, and evaluate mathematical expressions involving multi-dimensional arrays efficiently. It can use GPUs and perform efficient symbolic differentiation. |
|
||||
| [theano-scan](http://nbviewer.ipython.org/github/donnemartin/data-science-ipython-notebooks/blob/master/deep-learning/theano-tutorial/scan_tutorial/scan_tutorial.ipynb) | Learn scans, a mechanism to perform loops in a Theano graph. |
|
||||
| [theano-logistic](http://nbviewer.ipython.org/github/donnemartin/data-science-ipython-notebooks/blob/master/deep-learning/theano-tutorial/intro_theano/logistic_regression.ipynb) | Implement logistic regression in Theano. |
|
||||
| [theano-rnn](http://nbviewer.ipython.org/github/donnemartin/data-science-ipython-notebooks/blob/master/deep-learning/theano-tutorial/rnn_tutorial/simple_rnn.ipynb) | Implement recurrent neural networks in Theano. |
|
||||
| [theano-mlp](http://nbviewer.ipython.org/github/donnemartin/data-science-ipython-notebooks/blob/master/deep-learning/theano-tutorial/theano_mlp/theano_mlp.ipynb) | Implement multilayer perceptrons in Theano. |
|
||||
|
||||
### deep-learning-misc
|
||||
|
||||
| Notebook | Description |
|
||||
|--------------------------------------------------------------------------------------------------------------|-------------------------------------------------------------------------------------------------------------------------------------------------------------------|
|
||||
| [deep-dream](http://nbviewer.ipython.org/github/donnemartin/data-science-ipython-notebooks/blob/master/deep-learning/deep-dream/dream.ipynb) | Caffe-based computer vision program which uses a convolutional neural network to find and enhance patterns in images. |
|
||||
|
||||
<br/>
|
||||
<p align="center">
|
||||
|
|
|
|||
486
deep-learning/tensor-flow-examples/Setup_TensorFlow.md
Normal file
486
deep-learning/tensor-flow-examples/Setup_TensorFlow.md
Normal file
|
|
@ -0,0 +1,486 @@
|
|||
_From TensorFlow Official docs_
|
||||
|
||||
# Download and Setup <a class="md-anchor" id="AUTOGENERATED-download-and-setup"></a>
|
||||
|
||||
You can install TensorFlow using our provided binary packages or from source.
|
||||
|
||||
## Binary Installation <a class="md-anchor" id="AUTOGENERATED-binary-installation"></a>
|
||||
|
||||
The TensorFlow Python API currently requires Python 2.7: we are
|
||||
[working](https://github.com/tensorflow/tensorflow/issues/1) on adding support
|
||||
for Python 3.
|
||||
|
||||
The simplest way to install TensorFlow is using
|
||||
[pip](https://pypi.python.org/pypi/pip) for both Linux and Mac.
|
||||
|
||||
If you encounter installation errors, see
|
||||
[common problems](#common_install_problems) for some solutions. To simplify
|
||||
installation, please consider using our virtualenv-based instructions
|
||||
[here](#virtualenv_install).
|
||||
|
||||
### Ubuntu/Linux 64-bit <a class="md-anchor" id="AUTOGENERATED-ubuntu-linux-64-bit"></a>
|
||||
|
||||
```bash
|
||||
# For CPU-only version
|
||||
$ pip install https://storage.googleapis.com/tensorflow/linux/cpu/tensorflow-0.5.0-cp27-none-linux_x86_64.whl
|
||||
|
||||
# For GPU-enabled version (only install this version if you have the CUDA sdk installed)
|
||||
$ pip install https://storage.googleapis.com/tensorflow/linux/gpu/tensorflow-0.5.0-cp27-none-linux_x86_64.whl
|
||||
```
|
||||
|
||||
### Mac OS X <a class="md-anchor" id="AUTOGENERATED-mac-os-x"></a>
|
||||
|
||||
On OS X, we recommend installing [homebrew](http://brew.sh) and `brew install
|
||||
python` before proceeding, or installing TensorFlow within [virtualenv](#virtualenv_install).
|
||||
|
||||
```bash
|
||||
# Only CPU-version is available at the moment.
|
||||
$ pip install https://storage.googleapis.com/tensorflow/mac/tensorflow-0.5.0-py2-none-any.whl
|
||||
```
|
||||
|
||||
## Docker-based installation <a class="md-anchor" id="AUTOGENERATED-docker-based-installation"></a>
|
||||
|
||||
We also support running TensorFlow via [Docker](http://docker.com/), which lets
|
||||
you avoid worrying about setting up dependencies.
|
||||
|
||||
First, [install Docker](http://docs.docker.com/engine/installation/). Once
|
||||
Docker is up and running, you can start a container with one command:
|
||||
|
||||
```bash
|
||||
$ docker run -it b.gcr.io/tensorflow/tensorflow
|
||||
```
|
||||
|
||||
This will start a container with TensorFlow and all its dependencies already
|
||||
installed.
|
||||
|
||||
### Additional images <a class="md-anchor" id="AUTOGENERATED-additional-images"></a>
|
||||
|
||||
The default Docker image above contains just a minimal set of libraries for
|
||||
getting up and running with TensorFlow. We also have the following container,
|
||||
which you can use in the `docker run` command above:
|
||||
|
||||
* `b.gcr.io/tensorflow/tensorflow-full`: Contains a complete TensorFlow source
|
||||
installation, including all utilities needed to build and run TensorFlow. This
|
||||
makes it easy to experiment directly with the source, without needing to
|
||||
install any of the dependencies described above.
|
||||
|
||||
## VirtualEnv-based installation <a class="md-anchor" id="virtualenv_install"></a>
|
||||
|
||||
We recommend using [virtualenv](https://pypi.python.org/pypi/virtualenv) to
|
||||
create an isolated container and install TensorFlow in that container -- it is
|
||||
optional but makes verifying installation issues easier.
|
||||
|
||||
First, install all required tools:
|
||||
|
||||
```bash
|
||||
# On Linux:
|
||||
$ sudo apt-get install python-pip python-dev python-virtualenv
|
||||
|
||||
# On Mac:
|
||||
$ sudo easy_install pip # If pip is not already installed
|
||||
$ sudo pip install --upgrade virtualenv
|
||||
```
|
||||
|
||||
Next, set up a new virtualenv environment. To set it up in the
|
||||
directory `~/tensorflow`, run:
|
||||
|
||||
```bash
|
||||
$ virtualenv --system-site-packages ~/tensorflow
|
||||
$ cd ~/tensorflow
|
||||
```
|
||||
|
||||
Then activate the virtualenv:
|
||||
|
||||
```bash
|
||||
$ source bin/activate # If using bash
|
||||
$ source bin/activate.csh # If using csh
|
||||
(tensorflow)$ # Your prompt should change
|
||||
```
|
||||
|
||||
Inside the virtualenv, install TensorFlow:
|
||||
|
||||
```bash
|
||||
# For CPU-only linux x86_64 version
|
||||
(tensorflow)$ pip install --upgrade https://storage.googleapis.com/tensorflow/linux/cpu/tensorflow-0.5.0-cp27-none-linux_x86_64.whl
|
||||
|
||||
# For GPU-enabled linux x86_64 version
|
||||
(tensorflow)$ pip install --upgrade https://storage.googleapis.com/tensorflow/linux/gpu/tensorflow-0.5.0-cp27-none-linux_x86_64.whl
|
||||
|
||||
# For Mac CPU-only version
|
||||
(tensorflow)$ pip install --upgrade https://storage.googleapis.com/tensorflow/mac/tensorflow-0.5.0-py2-none-any.whl
|
||||
```
|
||||
|
||||
Make sure you have downloaded the source code for TensorFlow, and then you can
|
||||
then run an example TensorFlow program like:
|
||||
|
||||
```bash
|
||||
(tensorflow)$ cd tensorflow/models/image/mnist
|
||||
(tensorflow)$ python convolutional.py
|
||||
|
||||
# When you are done using TensorFlow:
|
||||
(tensorflow)$ deactivate # Deactivate the virtualenv
|
||||
|
||||
$ # Your prompt should change back
|
||||
```
|
||||
|
||||
## Try your first TensorFlow program <a class="md-anchor" id="AUTOGENERATED-try-your-first-tensorflow-program"></a>
|
||||
|
||||
### (Optional) Enable GPU Support <a class="md-anchor" id="AUTOGENERATED--optional--enable-gpu-support"></a>
|
||||
|
||||
If you installed the GPU-enabled TensorFlow pip binary, you must have the
|
||||
correct versions of the CUDA SDK and CUDNN installed on your
|
||||
system. Please see [the CUDA installation instructions](#install_cuda).
|
||||
|
||||
You also need to set the `LD_LIBRARY_PATH` and `CUDA_HOME` environment
|
||||
variables. Consider adding the commands below to your `~/.bash_profile`. These
|
||||
assume your CUDA installation is in `/usr/local/cuda`:
|
||||
|
||||
```bash
|
||||
export LD_LIBRARY_PATH="$LD_LIBRARY_PATH:/usr/local/cuda/lib64"
|
||||
export CUDA_HOME=/usr/local/cuda
|
||||
```
|
||||
|
||||
### Run TensorFlow <a class="md-anchor" id="AUTOGENERATED-run-tensorflow"></a>
|
||||
|
||||
Open a python terminal:
|
||||
|
||||
```bash
|
||||
$ python
|
||||
|
||||
>>> import tensorflow as tf
|
||||
>>> hello = tf.constant('Hello, TensorFlow!')
|
||||
>>> sess = tf.Session()
|
||||
>>> print sess.run(hello)
|
||||
Hello, TensorFlow!
|
||||
>>> a = tf.constant(10)
|
||||
>>> b = tf.constant(32)
|
||||
>>> print sess.run(a+b)
|
||||
42
|
||||
>>>
|
||||
|
||||
```
|
||||
|
||||
## Installing from sources <a class="md-anchor" id="source"></a>
|
||||
|
||||
### Clone the TensorFlow repository <a class="md-anchor" id="AUTOGENERATED-clone-the-tensorflow-repository"></a>
|
||||
|
||||
```bash
|
||||
$ git clone --recurse-submodules https://github.com/tensorflow/tensorflow
|
||||
```
|
||||
|
||||
`--recurse-submodules` is required to fetch the protobuf library that TensorFlow
|
||||
depends on.
|
||||
|
||||
### Installation for Linux <a class="md-anchor" id="AUTOGENERATED-installation-for-linux"></a>
|
||||
|
||||
#### Install Bazel <a class="md-anchor" id="AUTOGENERATED-install-bazel"></a>
|
||||
|
||||
|
||||
Follow instructions [here](http://bazel.io/docs/install.html) to install the
|
||||
dependencies for Bazel. Then download bazel version 0.1.1 using the
|
||||
[installer for your system](https://github.com/bazelbuild/bazel/releases) and
|
||||
run the installer as mentioned there:
|
||||
|
||||
```bash
|
||||
$ chmod +x PATH_TO_INSTALL.SH
|
||||
$ ./PATH_TO_INSTALL.SH --user
|
||||
```
|
||||
|
||||
Remember to replace `PATH_TO_INSTALL.SH` to point to the location where you
|
||||
downloaded the installer.
|
||||
|
||||
Finally, follow the instructions in that script to place bazel into your binary
|
||||
path.
|
||||
|
||||
#### Install other dependencies <a class="md-anchor" id="AUTOGENERATED-install-other-dependencies"></a>
|
||||
|
||||
```bash
|
||||
$ sudo apt-get install python-numpy swig python-dev
|
||||
```
|
||||
|
||||
#### Optional: Install CUDA (GPUs on Linux) <a class="md-anchor" id="install_cuda"></a>
|
||||
|
||||
In order to build or run TensorFlow with GPU support, both Cuda Toolkit 7.0 and
|
||||
CUDNN 6.5 V2 from NVIDIA need to be installed.
|
||||
|
||||
TensorFlow GPU support requires having a GPU card with NVidia Compute Capability >= 3.5. Supported cards include but are not limited to:
|
||||
|
||||
* NVidia Titan
|
||||
* NVidia Titan X
|
||||
* NVidia K20
|
||||
* NVidia K40
|
||||
|
||||
##### Download and install Cuda Toolkit 7.0 <a class="md-anchor" id="AUTOGENERATED-download-and-install-cuda-toolkit-7.0"></a>
|
||||
|
||||
https://developer.nvidia.com/cuda-toolkit-70
|
||||
|
||||
Install the toolkit into e.g. `/usr/local/cuda`
|
||||
|
||||
##### Download and install CUDNN Toolkit 6.5 <a class="md-anchor" id="AUTOGENERATED-download-and-install-cudnn-toolkit-6.5"></a>
|
||||
|
||||
https://developer.nvidia.com/rdp/cudnn-archive
|
||||
|
||||
Uncompress and copy the cudnn files into the toolkit directory. Assuming the
|
||||
toolkit is installed in `/usr/local/cuda`:
|
||||
|
||||
``` bash
|
||||
tar xvzf cudnn-6.5-linux-x64-v2.tgz
|
||||
sudo cp cudnn-6.5-linux-x64-v2/cudnn.h /usr/local/cuda/include
|
||||
sudo cp cudnn-6.5-linux-x64-v2/libcudnn* /usr/local/cuda/lib64
|
||||
```
|
||||
|
||||
##### Configure TensorFlow's canonical view of Cuda libraries <a class="md-anchor" id="AUTOGENERATED-configure-tensorflow-s-canonical-view-of-cuda-libraries"></a>
|
||||
From the root of your source tree, run:
|
||||
|
||||
``` bash
|
||||
$ ./configure
|
||||
Do you wish to build TensorFlow with GPU support? [y/n] y
|
||||
GPU support will be enabled for TensorFlow
|
||||
|
||||
Please specify the location where CUDA 7.0 toolkit is installed. Refer to
|
||||
README.md for more details. [default is: /usr/local/cuda]: /usr/local/cuda
|
||||
|
||||
Please specify the location where CUDNN 6.5 V2 library is installed. Refer to
|
||||
README.md for more details. [default is: /usr/local/cuda]: /usr/local/cuda
|
||||
|
||||
Setting up Cuda include
|
||||
Setting up Cuda lib64
|
||||
Setting up Cuda bin
|
||||
Setting up Cuda nvvm
|
||||
Configuration finished
|
||||
```
|
||||
|
||||
This creates a canonical set of symbolic links to the Cuda libraries on your system.
|
||||
Every time you change the Cuda library paths you need to run this step again before
|
||||
you invoke the bazel build command.
|
||||
|
||||
##### Build your target with GPU support. <a class="md-anchor" id="AUTOGENERATED-build-your-target-with-gpu-support."></a>
|
||||
From the root of your source tree, run:
|
||||
|
||||
```bash
|
||||
$ bazel build -c opt --config=cuda //tensorflow/cc:tutorials_example_trainer
|
||||
|
||||
$ bazel-bin/tensorflow/cc/tutorials_example_trainer --use_gpu
|
||||
# Lots of output. This tutorial iteratively calculates the major eigenvalue of
|
||||
# a 2x2 matrix, on GPU. The last few lines look like this.
|
||||
000009/000005 lambda = 2.000000 x = [0.894427 -0.447214] y = [1.788854 -0.894427]
|
||||
000006/000001 lambda = 2.000000 x = [0.894427 -0.447214] y = [1.788854 -0.894427]
|
||||
000009/000009 lambda = 2.000000 x = [0.894427 -0.447214] y = [1.788854 -0.894427]
|
||||
```
|
||||
|
||||
Note that "--config=cuda" is needed to enable the GPU support.
|
||||
|
||||
##### Enabling Cuda 3.0. <a class="md-anchor" id="AUTOGENERATED-enabling-cuda-3.0."></a>
|
||||
TensorFlow officially supports Cuda devices with 3.5 and 5.2 compute
|
||||
capabilities. In order to enable earlier Cuda devices such as Grid K520, you
|
||||
need to target Cuda 3.0. This can be done through TensorFlow unofficial
|
||||
settings with "configure".
|
||||
|
||||
```bash
|
||||
$ TF_UNOFFICIAL_SETTING=1 ./configure
|
||||
|
||||
# Same as the official settings above
|
||||
|
||||
WARNING: You are configuring unofficial settings in TensorFlow. Because some
|
||||
external libraries are not backward compatible, these settings are largely
|
||||
untested and unsupported.
|
||||
|
||||
Please specify a list of comma-separated Cuda compute capabilities you want to
|
||||
build with. You can find the compute capability of your device at:
|
||||
https://developer.nvidia.com/cuda-gpus.
|
||||
Please note that each additional compute capability significantly increases
|
||||
your build time and binary size. [Default is: "3.5,5.2"]: 3.0
|
||||
|
||||
Setting up Cuda include
|
||||
Setting up Cuda lib64
|
||||
Setting up Cuda bin
|
||||
Setting up Cuda nvvm
|
||||
Configuration finished
|
||||
```
|
||||
|
||||
##### Known issues <a class="md-anchor" id="AUTOGENERATED-known-issues"></a>
|
||||
|
||||
* Although it is possible to build both Cuda and non-Cuda configs under the same
|
||||
source tree, we recommend to run "bazel clean" when switching between these two
|
||||
configs in the same source tree.
|
||||
|
||||
* You have to run configure before running bazel build. Otherwise, the build
|
||||
will fail with a clear error message. In the future, we might consider making
|
||||
this more conveninent by including the configure step in our build process,
|
||||
given necessary bazel new feature support.
|
||||
|
||||
### Installation for Mac OS X <a class="md-anchor" id="AUTOGENERATED-installation-for-mac-os-x"></a>
|
||||
|
||||
Mac needs the same set of dependencies as Linux, however their installing those
|
||||
dependencies is different. Here is a set of useful links to help with installing
|
||||
the dependencies on Mac OS X :
|
||||
|
||||
#### Bazel <a class="md-anchor" id="AUTOGENERATED-bazel"></a>
|
||||
|
||||
Look for installation instructions for Mac OS X on
|
||||
[this](http://bazel.io/docs/install.html) page.
|
||||
|
||||
#### SWIG <a class="md-anchor" id="AUTOGENERATED-swig"></a>
|
||||
|
||||
[Mac OS X installation](http://www.swig.org/Doc3.0/Preface.html#Preface_osx_installation).
|
||||
|
||||
Notes : You need to install
|
||||
[PCRE](ftp://ftp.csx.cam.ac.uk/pub/software/programming/pcre/) and *NOT* PCRE2.
|
||||
|
||||
#### Numpy <a class="md-anchor" id="AUTOGENERATED-numpy"></a>
|
||||
|
||||
Follow installation instructions [here](http://docs.scipy.org/doc/numpy/user/install.html).
|
||||
|
||||
|
||||
### Create the pip package and install <a class="md-anchor" id="create-pip"></a>
|
||||
|
||||
```bash
|
||||
$ bazel build -c opt //tensorflow/tools/pip_package:build_pip_package
|
||||
|
||||
# To build with GPU support:
|
||||
$ bazel build -c opt --config=cuda //tensorflow/tools/pip_package:build_pip_package
|
||||
|
||||
$ bazel-bin/tensorflow/tools/pip_package/build_pip_package /tmp/tensorflow_pkg
|
||||
|
||||
# The name of the .whl file will depend on your platform.
|
||||
$ pip install /tmp/tensorflow_pkg/tensorflow-0.5.0-cp27-none-linux_x86_64.whl
|
||||
```
|
||||
|
||||
## Train your first TensorFlow neural net model <a class="md-anchor" id="AUTOGENERATED-train-your-first-tensorflow-neural-net-model"></a>
|
||||
|
||||
Starting from the root of your source tree, run:
|
||||
|
||||
```python
|
||||
$ cd tensorflow/models/image/mnist
|
||||
$ python convolutional.py
|
||||
Succesfully downloaded train-images-idx3-ubyte.gz 9912422 bytes.
|
||||
Succesfully downloaded train-labels-idx1-ubyte.gz 28881 bytes.
|
||||
Succesfully downloaded t10k-images-idx3-ubyte.gz 1648877 bytes.
|
||||
Succesfully downloaded t10k-labels-idx1-ubyte.gz 4542 bytes.
|
||||
Extracting data/train-images-idx3-ubyte.gz
|
||||
Extracting data/train-labels-idx1-ubyte.gz
|
||||
Extracting data/t10k-images-idx3-ubyte.gz
|
||||
Extracting data/t10k-labels-idx1-ubyte.gz
|
||||
Initialized!
|
||||
Epoch 0.00
|
||||
Minibatch loss: 12.054, learning rate: 0.010000
|
||||
Minibatch error: 90.6%
|
||||
Validation error: 84.6%
|
||||
Epoch 0.12
|
||||
Minibatch loss: 3.285, learning rate: 0.010000
|
||||
Minibatch error: 6.2%
|
||||
Validation error: 7.0%
|
||||
...
|
||||
...
|
||||
```
|
||||
|
||||
## Common Problems <a class="md-anchor" id="common_install_problems"></a>
|
||||
|
||||
### GPU-related issues <a class="md-anchor" id="AUTOGENERATED-gpu-related-issues"></a>
|
||||
|
||||
If you encounter the following when trying to run a TensorFlow program:
|
||||
|
||||
```python
|
||||
ImportError: libcudart.so.7.0: cannot open shared object file: No such file or directory
|
||||
```
|
||||
|
||||
Make sure you followed the the GPU installation [instructions](#install_cuda).
|
||||
|
||||
### Pip installation issues <a class="md-anchor" id="AUTOGENERATED-pip-installation-issues"></a>
|
||||
|
||||
#### Can't find setup.py <a class="md-anchor" id="AUTOGENERATED-can-t-find-setup.py"></a>
|
||||
|
||||
If, during `pip install`, you encounter an error like:
|
||||
|
||||
```bash
|
||||
...
|
||||
IOError: [Errno 2] No such file or directory: '/tmp/pip-o6Tpui-build/setup.py'
|
||||
```
|
||||
|
||||
Solution: upgrade your version of `pip`:
|
||||
|
||||
```bash
|
||||
pip install --upgrade pip
|
||||
```
|
||||
|
||||
This may require `sudo`, depending on how `pip` is installed.
|
||||
|
||||
#### SSLError: SSL_VERIFY_FAILED <a class="md-anchor" id="AUTOGENERATED-sslerror--ssl_verify_failed"></a>
|
||||
|
||||
If, during pip install from a URL, you encounter an error like:
|
||||
|
||||
```bash
|
||||
...
|
||||
SSLError: [SSL: CERTIFICATE_VERIFY_FAILED] certificate verify failed
|
||||
```
|
||||
|
||||
Solution: Download the wheel manually via curl or wget, and pip install locally.
|
||||
|
||||
### On Linux <a class="md-anchor" id="AUTOGENERATED-on-linux"></a>
|
||||
|
||||
If you encounter:
|
||||
|
||||
```python
|
||||
...
|
||||
"__add__", "__radd__",
|
||||
^
|
||||
SyntaxError: invalid syntax
|
||||
```
|
||||
|
||||
Solution: make sure you are using Python 2.7.
|
||||
|
||||
### On MacOSX <a class="md-anchor" id="AUTOGENERATED-on-macosx"></a>
|
||||
|
||||
|
||||
If you encounter:
|
||||
|
||||
```python
|
||||
import six.moves.copyreg as copyreg
|
||||
|
||||
ImportError: No module named copyreg
|
||||
```
|
||||
|
||||
Solution: TensorFlow depends on protobuf, which requires `six-1.10.0`. Apple's
|
||||
default python environment has `six-1.4.1` and may be difficult to upgrade.
|
||||
There are several ways to fix this:
|
||||
|
||||
1. Upgrade the system-wide copy of `six`:
|
||||
|
||||
```bash
|
||||
sudo easy_install -U six
|
||||
```
|
||||
|
||||
2. Install a separate copy of python via homebrew:
|
||||
|
||||
```bash
|
||||
brew install python
|
||||
```
|
||||
|
||||
3. Build or use TensorFlow
|
||||
[within `virtualenv`](#virtualenv_install).
|
||||
|
||||
|
||||
|
||||
If you encounter:
|
||||
|
||||
```
|
||||
>>> import tensorflow as tf
|
||||
Traceback (most recent call last):
|
||||
File "<stdin>", line 1, in <module>
|
||||
File "/usr/local/lib/python2.7/site-packages/tensorflow/__init__.py", line 4, in <module>
|
||||
from tensorflow.python import *
|
||||
File "/usr/local/lib/python2.7/site-packages/tensorflow/python/__init__.py", line 13, in <module>
|
||||
from tensorflow.core.framework.graph_pb2 import *
|
||||
...
|
||||
File "/usr/local/lib/python2.7/site-packages/tensorflow/core/framework/tensor_shape_pb2.py", line 22, in <module>
|
||||
serialized_pb=_b('\n,tensorflow/core/framework/tensor_shape.proto\x12\ntensorflow\"d\n\x10TensorShapeProto\x12-\n\x03\x64im\x18\x02 \x03(\x0b\x32 .tensorflow.TensorShapeProto.Dim\x1a!\n\x03\x44im\x12\x0c\n\x04size\x18\x01 \x01(\x03\x12\x0c\n\x04name\x18\x02 \x01(\tb\x06proto3')
|
||||
TypeError: __init__() got an unexpected keyword argument 'syntax'
|
||||
```
|
||||
|
||||
This is due to a conflict between protobuf versions (we require protobuf 3.0.0).
|
||||
The best current solution is to make sure older versions of protobuf are not
|
||||
installed, such as:
|
||||
|
||||
```bash
|
||||
brew reinstall --devel protobuf
|
||||
```
|
||||
144
deep-learning/tensor-flow-examples/input_data.py
Normal file
144
deep-learning/tensor-flow-examples/input_data.py
Normal file
|
|
@ -0,0 +1,144 @@
|
|||
"""Functions for downloading and reading MNIST data."""
|
||||
from __future__ import print_function
|
||||
import gzip
|
||||
import os
|
||||
import urllib
|
||||
import numpy
|
||||
SOURCE_URL = 'http://yann.lecun.com/exdb/mnist/'
|
||||
def maybe_download(filename, work_directory):
|
||||
"""Download the data from Yann's website, unless it's already here."""
|
||||
if not os.path.exists(work_directory):
|
||||
os.mkdir(work_directory)
|
||||
filepath = os.path.join(work_directory, filename)
|
||||
if not os.path.exists(filepath):
|
||||
filepath, _ = urllib.urlretrieve(SOURCE_URL + filename, filepath)
|
||||
statinfo = os.stat(filepath)
|
||||
print('Succesfully downloaded', filename, statinfo.st_size, 'bytes.')
|
||||
return filepath
|
||||
def _read32(bytestream):
|
||||
dt = numpy.dtype(numpy.uint32).newbyteorder('>')
|
||||
return numpy.frombuffer(bytestream.read(4), dtype=dt)
|
||||
def extract_images(filename):
|
||||
"""Extract the images into a 4D uint8 numpy array [index, y, x, depth]."""
|
||||
print('Extracting', filename)
|
||||
with gzip.open(filename) as bytestream:
|
||||
magic = _read32(bytestream)
|
||||
if magic != 2051:
|
||||
raise ValueError(
|
||||
'Invalid magic number %d in MNIST image file: %s' %
|
||||
(magic, filename))
|
||||
num_images = _read32(bytestream)
|
||||
rows = _read32(bytestream)
|
||||
cols = _read32(bytestream)
|
||||
buf = bytestream.read(rows * cols * num_images)
|
||||
data = numpy.frombuffer(buf, dtype=numpy.uint8)
|
||||
data = data.reshape(num_images, rows, cols, 1)
|
||||
return data
|
||||
def dense_to_one_hot(labels_dense, num_classes=10):
|
||||
"""Convert class labels from scalars to one-hot vectors."""
|
||||
num_labels = labels_dense.shape[0]
|
||||
index_offset = numpy.arange(num_labels) * num_classes
|
||||
labels_one_hot = numpy.zeros((num_labels, num_classes))
|
||||
labels_one_hot.flat[index_offset + labels_dense.ravel()] = 1
|
||||
return labels_one_hot
|
||||
def extract_labels(filename, one_hot=False):
|
||||
"""Extract the labels into a 1D uint8 numpy array [index]."""
|
||||
print('Extracting', filename)
|
||||
with gzip.open(filename) as bytestream:
|
||||
magic = _read32(bytestream)
|
||||
if magic != 2049:
|
||||
raise ValueError(
|
||||
'Invalid magic number %d in MNIST label file: %s' %
|
||||
(magic, filename))
|
||||
num_items = _read32(bytestream)
|
||||
buf = bytestream.read(num_items)
|
||||
labels = numpy.frombuffer(buf, dtype=numpy.uint8)
|
||||
if one_hot:
|
||||
return dense_to_one_hot(labels)
|
||||
return labels
|
||||
class DataSet(object):
|
||||
def __init__(self, images, labels, fake_data=False):
|
||||
if fake_data:
|
||||
self._num_examples = 10000
|
||||
else:
|
||||
assert images.shape[0] == labels.shape[0], (
|
||||
"images.shape: %s labels.shape: %s" % (images.shape,
|
||||
labels.shape))
|
||||
self._num_examples = images.shape[0]
|
||||
# Convert shape from [num examples, rows, columns, depth]
|
||||
# to [num examples, rows*columns] (assuming depth == 1)
|
||||
assert images.shape[3] == 1
|
||||
images = images.reshape(images.shape[0],
|
||||
images.shape[1] * images.shape[2])
|
||||
# Convert from [0, 255] -> [0.0, 1.0].
|
||||
images = images.astype(numpy.float32)
|
||||
images = numpy.multiply(images, 1.0 / 255.0)
|
||||
self._images = images
|
||||
self._labels = labels
|
||||
self._epochs_completed = 0
|
||||
self._index_in_epoch = 0
|
||||
@property
|
||||
def images(self):
|
||||
return self._images
|
||||
@property
|
||||
def labels(self):
|
||||
return self._labels
|
||||
@property
|
||||
def num_examples(self):
|
||||
return self._num_examples
|
||||
@property
|
||||
def epochs_completed(self):
|
||||
return self._epochs_completed
|
||||
def next_batch(self, batch_size, fake_data=False):
|
||||
"""Return the next `batch_size` examples from this data set."""
|
||||
if fake_data:
|
||||
fake_image = [1.0 for _ in xrange(784)]
|
||||
fake_label = 0
|
||||
return [fake_image for _ in xrange(batch_size)], [
|
||||
fake_label for _ in xrange(batch_size)]
|
||||
start = self._index_in_epoch
|
||||
self._index_in_epoch += batch_size
|
||||
if self._index_in_epoch > self._num_examples:
|
||||
# Finished epoch
|
||||
self._epochs_completed += 1
|
||||
# Shuffle the data
|
||||
perm = numpy.arange(self._num_examples)
|
||||
numpy.random.shuffle(perm)
|
||||
self._images = self._images[perm]
|
||||
self._labels = self._labels[perm]
|
||||
# Start next epoch
|
||||
start = 0
|
||||
self._index_in_epoch = batch_size
|
||||
assert batch_size <= self._num_examples
|
||||
end = self._index_in_epoch
|
||||
return self._images[start:end], self._labels[start:end]
|
||||
def read_data_sets(train_dir, fake_data=False, one_hot=False):
|
||||
class DataSets(object):
|
||||
pass
|
||||
data_sets = DataSets()
|
||||
if fake_data:
|
||||
data_sets.train = DataSet([], [], fake_data=True)
|
||||
data_sets.validation = DataSet([], [], fake_data=True)
|
||||
data_sets.test = DataSet([], [], fake_data=True)
|
||||
return data_sets
|
||||
TRAIN_IMAGES = 'train-images-idx3-ubyte.gz'
|
||||
TRAIN_LABELS = 'train-labels-idx1-ubyte.gz'
|
||||
TEST_IMAGES = 't10k-images-idx3-ubyte.gz'
|
||||
TEST_LABELS = 't10k-labels-idx1-ubyte.gz'
|
||||
VALIDATION_SIZE = 5000
|
||||
local_file = maybe_download(TRAIN_IMAGES, train_dir)
|
||||
train_images = extract_images(local_file)
|
||||
local_file = maybe_download(TRAIN_LABELS, train_dir)
|
||||
train_labels = extract_labels(local_file, one_hot=one_hot)
|
||||
local_file = maybe_download(TEST_IMAGES, train_dir)
|
||||
test_images = extract_images(local_file)
|
||||
local_file = maybe_download(TEST_LABELS, train_dir)
|
||||
test_labels = extract_labels(local_file, one_hot=one_hot)
|
||||
validation_images = train_images[:VALIDATION_SIZE]
|
||||
validation_labels = train_labels[:VALIDATION_SIZE]
|
||||
train_images = train_images[VALIDATION_SIZE:]
|
||||
train_labels = train_labels[VALIDATION_SIZE:]
|
||||
data_sets.train = DataSet(train_images, train_labels)
|
||||
data_sets.validation = DataSet(validation_images, validation_labels)
|
||||
data_sets.test = DataSet(test_images, test_labels)
|
||||
return data_sets
|
||||
85
deep-learning/tensor-flow-examples/multigpu_basics.py
Normal file
85
deep-learning/tensor-flow-examples/multigpu_basics.py
Normal file
|
|
@ -0,0 +1,85 @@
|
|||
#Multi GPU Basic example
|
||||
'''
|
||||
This tutorial requires your machine to have 2 GPUs
|
||||
"/cpu:0": The CPU of your machine.
|
||||
"/gpu:0": The first GPU of your machine
|
||||
"/gpu:1": The second GPU of your machine
|
||||
'''
|
||||
|
||||
import numpy as np
|
||||
import tensorflow as tf
|
||||
import datetime
|
||||
|
||||
#Processing Units logs
|
||||
log_device_placement = True
|
||||
|
||||
#num of multiplications to perform
|
||||
n = 10
|
||||
|
||||
'''
|
||||
Example: compute A^n + B^n on 2 GPUs
|
||||
Results on 8 cores with 2 GTX-980:
|
||||
* Single GPU computation time: 0:00:11.277449
|
||||
* Multi GPU computation time: 0:00:07.131701
|
||||
'''
|
||||
#Create random large matrix
|
||||
A = np.random.rand(1e4, 1e4).astype('float32')
|
||||
B = np.random.rand(1e4, 1e4).astype('float32')
|
||||
|
||||
# Creates a graph to store results
|
||||
c1 = []
|
||||
c2 = []
|
||||
|
||||
def matpow(M, n):
|
||||
if n < 1: #Abstract cases where n < 1
|
||||
return M
|
||||
else:
|
||||
return tf.matmul(M, matpow(M, n-1))
|
||||
|
||||
'''
|
||||
Single GPU computing
|
||||
'''
|
||||
with tf.device('/gpu:0'):
|
||||
a = tf.constant(A)
|
||||
b = tf.constant(B)
|
||||
#compute A^n and B^n and store results in c1
|
||||
c1.append(matpow(a, n))
|
||||
c1.append(matpow(b, n))
|
||||
|
||||
with tf.device('/cpu:0'):
|
||||
sum = tf.add_n(c1) #Addition of all elements in c1, i.e. A^n + B^n
|
||||
|
||||
t1_1 = datetime.datetime.now()
|
||||
with tf.Session(config=tf.ConfigProto(log_device_placement=log_device_placement)) as sess:
|
||||
# Runs the op.
|
||||
sess.run(sum)
|
||||
t2_1 = datetime.datetime.now()
|
||||
|
||||
|
||||
'''
|
||||
Multi GPU computing
|
||||
'''
|
||||
#GPU:0 computes A^n
|
||||
with tf.device('/gpu:0'):
|
||||
#compute A^n and store result in c2
|
||||
a = tf.constant(A)
|
||||
c2.append(matpow(a, n))
|
||||
|
||||
#GPU:1 computes B^n
|
||||
with tf.device('/gpu:1'):
|
||||
#compute B^n and store result in c2
|
||||
b = tf.constant(B)
|
||||
c2.append(matpow(b, n))
|
||||
|
||||
with tf.device('/cpu:0'):
|
||||
sum = tf.add_n(c2) #Addition of all elements in c2, i.e. A^n + B^n
|
||||
|
||||
t1_2 = datetime.datetime.now()
|
||||
with tf.Session(config=tf.ConfigProto(log_device_placement=log_device_placement)) as sess:
|
||||
# Runs the op.
|
||||
sess.run(sum)
|
||||
t2_2 = datetime.datetime.now()
|
||||
|
||||
|
||||
print "Single GPU computation time: " + str(t2_1-t1_1)
|
||||
print "Multi GPU computation time: " + str(t2_2-t1_2)
|
||||
|
|
@ -0,0 +1,220 @@
|
|||
{
|
||||
"cells": [
|
||||
{
|
||||
"cell_type": "markdown",
|
||||
"metadata": {},
|
||||
"source": [
|
||||
"# Basic Operations in TensorFlow\n",
|
||||
"\n",
|
||||
"Credits: Forked from [TensorFlow-Examples](https://github.com/aymericdamien/TensorFlow-Examples) by Aymeric Damien\n",
|
||||
"\n",
|
||||
"## Setup\n",
|
||||
"\n",
|
||||
"Refer to the [setup instructions](http://nbviewer.ipython.org/github/donnemartin/data-science-ipython-notebooks/blob/master/deep-learning/tensor-flow-examples/Setup_TensorFlow.md)"
|
||||
]
|
||||
},
|
||||
{
|
||||
"cell_type": "code",
|
||||
"execution_count": 2,
|
||||
"metadata": {
|
||||
"collapsed": true
|
||||
},
|
||||
"outputs": [],
|
||||
"source": [
|
||||
"import tensorflow as tf"
|
||||
]
|
||||
},
|
||||
{
|
||||
"cell_type": "code",
|
||||
"execution_count": 3,
|
||||
"metadata": {
|
||||
"collapsed": true
|
||||
},
|
||||
"outputs": [],
|
||||
"source": [
|
||||
"# Basic constant operations\n",
|
||||
"# The value returned by the constructor represents the output\n",
|
||||
"# of the Constant op.\n",
|
||||
"a = tf.constant(2)\n",
|
||||
"b = tf.constant(3)"
|
||||
]
|
||||
},
|
||||
{
|
||||
"cell_type": "code",
|
||||
"execution_count": 4,
|
||||
"metadata": {
|
||||
"collapsed": false
|
||||
},
|
||||
"outputs": [
|
||||
{
|
||||
"name": "stdout",
|
||||
"output_type": "stream",
|
||||
"text": [
|
||||
"a=2, b=3\n",
|
||||
"Addition with constants: 5\n",
|
||||
"Multiplication with constants: 6\n"
|
||||
]
|
||||
}
|
||||
],
|
||||
"source": [
|
||||
"# Launch the default graph.\n",
|
||||
"with tf.Session() as sess:\n",
|
||||
" print \"a=2, b=3\"\n",
|
||||
" print \"Addition with constants: %i\" % sess.run(a+b)\n",
|
||||
" print \"Multiplication with constants: %i\" % sess.run(a*b)"
|
||||
]
|
||||
},
|
||||
{
|
||||
"cell_type": "code",
|
||||
"execution_count": 5,
|
||||
"metadata": {
|
||||
"collapsed": true
|
||||
},
|
||||
"outputs": [],
|
||||
"source": [
|
||||
"# Basic Operations with variable as graph input\n",
|
||||
"# The value returned by the constructor represents the output\n",
|
||||
"# of the Variable op. (define as input when running session)\n",
|
||||
"# tf Graph input\n",
|
||||
"a = tf.placeholder(tf.types.int16)\n",
|
||||
"b = tf.placeholder(tf.types.int16)"
|
||||
]
|
||||
},
|
||||
{
|
||||
"cell_type": "code",
|
||||
"execution_count": 6,
|
||||
"metadata": {
|
||||
"collapsed": true
|
||||
},
|
||||
"outputs": [],
|
||||
"source": [
|
||||
"# Define some operations\n",
|
||||
"add = tf.add(a, b)\n",
|
||||
"mul = tf.mul(a, b)"
|
||||
]
|
||||
},
|
||||
{
|
||||
"cell_type": "code",
|
||||
"execution_count": 7,
|
||||
"metadata": {
|
||||
"collapsed": false
|
||||
},
|
||||
"outputs": [
|
||||
{
|
||||
"name": "stdout",
|
||||
"output_type": "stream",
|
||||
"text": [
|
||||
"Addition with variables: 5\n",
|
||||
"Multiplication with variables: 6\n"
|
||||
]
|
||||
}
|
||||
],
|
||||
"source": [
|
||||
"# Launch the default graph.\n",
|
||||
"with tf.Session() as sess:\n",
|
||||
" # Run every operation with variable input\n",
|
||||
" print \"Addition with variables: %i\" % sess.run(add, feed_dict={a: 2, b: 3})\n",
|
||||
" print \"Multiplication with variables: %i\" % sess.run(mul, feed_dict={a: 2, b: 3})"
|
||||
]
|
||||
},
|
||||
{
|
||||
"cell_type": "code",
|
||||
"execution_count": 8,
|
||||
"metadata": {
|
||||
"collapsed": true
|
||||
},
|
||||
"outputs": [],
|
||||
"source": [
|
||||
"# ----------------\n",
|
||||
"# More in details:\n",
|
||||
"# Matrix Multiplication from TensorFlow official tutorial\n",
|
||||
"\n",
|
||||
"# Create a Constant op that produces a 1x2 matrix. The op is\n",
|
||||
"# added as a node to the default graph.\n",
|
||||
"#\n",
|
||||
"# The value returned by the constructor represents the output\n",
|
||||
"# of the Constant op.\n",
|
||||
"matrix1 = tf.constant([[3., 3.]])"
|
||||
]
|
||||
},
|
||||
{
|
||||
"cell_type": "code",
|
||||
"execution_count": 9,
|
||||
"metadata": {
|
||||
"collapsed": true
|
||||
},
|
||||
"outputs": [],
|
||||
"source": [
|
||||
"# Create another Constant that produces a 2x1 matrix.\n",
|
||||
"matrix2 = tf.constant([[2.],[2.]])"
|
||||
]
|
||||
},
|
||||
{
|
||||
"cell_type": "code",
|
||||
"execution_count": 10,
|
||||
"metadata": {
|
||||
"collapsed": true
|
||||
},
|
||||
"outputs": [],
|
||||
"source": [
|
||||
"# Create a Matmul op that takes 'matrix1' and 'matrix2' as inputs.\n",
|
||||
"# The returned value, 'product', represents the result of the matrix\n",
|
||||
"# multiplication.\n",
|
||||
"product = tf.matmul(matrix1, matrix2)"
|
||||
]
|
||||
},
|
||||
{
|
||||
"cell_type": "code",
|
||||
"execution_count": 11,
|
||||
"metadata": {
|
||||
"collapsed": false
|
||||
},
|
||||
"outputs": [
|
||||
{
|
||||
"name": "stdout",
|
||||
"output_type": "stream",
|
||||
"text": [
|
||||
"[[ 12.]]\n"
|
||||
]
|
||||
}
|
||||
],
|
||||
"source": [
|
||||
"# To run the matmul op we call the session 'run()' method, passing 'product'\n",
|
||||
"# which represents the output of the matmul op. This indicates to the call\n",
|
||||
"# that we want to get the output of the matmul op back.\n",
|
||||
"#\n",
|
||||
"# All inputs needed by the op are run automatically by the session. They\n",
|
||||
"# typically are run in parallel.\n",
|
||||
"#\n",
|
||||
"# The call 'run(product)' thus causes the execution of threes ops in the\n",
|
||||
"# graph: the two constants and matmul.\n",
|
||||
"#\n",
|
||||
"# The output of the op is returned in 'result' as a numpy `ndarray` object.\n",
|
||||
"with tf.Session() as sess:\n",
|
||||
" result = sess.run(product)\n",
|
||||
" print result"
|
||||
]
|
||||
}
|
||||
],
|
||||
"metadata": {
|
||||
"kernelspec": {
|
||||
"display_name": "Python 3",
|
||||
"language": "python",
|
||||
"name": "python3"
|
||||
},
|
||||
"language_info": {
|
||||
"codemirror_mode": {
|
||||
"name": "ipython",
|
||||
"version": 3
|
||||
},
|
||||
"file_extension": ".py",
|
||||
"mimetype": "text/x-python",
|
||||
"name": "python",
|
||||
"nbconvert_exporter": "python",
|
||||
"pygments_lexer": "ipython3",
|
||||
"version": "3.4.3"
|
||||
}
|
||||
},
|
||||
"nbformat": 4,
|
||||
"nbformat_minor": 0
|
||||
}
|
||||
File diff suppressed because one or more lines are too long
|
|
@ -0,0 +1,227 @@
|
|||
{
|
||||
"cells": [
|
||||
{
|
||||
"cell_type": "markdown",
|
||||
"metadata": {
|
||||
"collapsed": true
|
||||
},
|
||||
"source": [
|
||||
"# Logistic Regression in TensorFlow\n",
|
||||
"\n",
|
||||
"Credits: Forked from [TensorFlow-Examples](https://github.com/aymericdamien/TensorFlow-Examples) by Aymeric Damien\n",
|
||||
"\n",
|
||||
"## Setup\n",
|
||||
"\n",
|
||||
"Refer to the [setup instructions](http://nbviewer.ipython.org/github/donnemartin/data-science-ipython-notebooks/blob/master/deep-learning/tensor-flow-examples/Setup_TensorFlow.md)"
|
||||
]
|
||||
},
|
||||
{
|
||||
"cell_type": "code",
|
||||
"execution_count": 5,
|
||||
"metadata": {
|
||||
"collapsed": false
|
||||
},
|
||||
"outputs": [
|
||||
{
|
||||
"name": "stdout",
|
||||
"output_type": "stream",
|
||||
"text": [
|
||||
"Extracting /tmp/data/train-images-idx3-ubyte.gz\n",
|
||||
"Extracting /tmp/data/train-labels-idx1-ubyte.gz\n",
|
||||
"Extracting /tmp/data/t10k-images-idx3-ubyte.gz\n",
|
||||
"Extracting /tmp/data/t10k-labels-idx1-ubyte.gz\n"
|
||||
]
|
||||
}
|
||||
],
|
||||
"source": [
|
||||
"# Import MINST data\n",
|
||||
"import input_data\n",
|
||||
"mnist = input_data.read_data_sets(\"/tmp/data/\", one_hot=True)"
|
||||
]
|
||||
},
|
||||
{
|
||||
"cell_type": "code",
|
||||
"execution_count": 6,
|
||||
"metadata": {
|
||||
"collapsed": true
|
||||
},
|
||||
"outputs": [],
|
||||
"source": [
|
||||
"import tensorflow as tf"
|
||||
]
|
||||
},
|
||||
{
|
||||
"cell_type": "code",
|
||||
"execution_count": 7,
|
||||
"metadata": {
|
||||
"collapsed": true
|
||||
},
|
||||
"outputs": [],
|
||||
"source": [
|
||||
"# Parameters\n",
|
||||
"learning_rate = 0.01\n",
|
||||
"training_epochs = 25\n",
|
||||
"batch_size = 100\n",
|
||||
"display_step = 1"
|
||||
]
|
||||
},
|
||||
{
|
||||
"cell_type": "code",
|
||||
"execution_count": 8,
|
||||
"metadata": {
|
||||
"collapsed": true
|
||||
},
|
||||
"outputs": [],
|
||||
"source": [
|
||||
"# tf Graph Input\n",
|
||||
"x = tf.placeholder(\"float\", [None, 784]) # mnist data image of shape 28*28=784\n",
|
||||
"y = tf.placeholder(\"float\", [None, 10]) # 0-9 digits recognition => 10 classes"
|
||||
]
|
||||
},
|
||||
{
|
||||
"cell_type": "code",
|
||||
"execution_count": 9,
|
||||
"metadata": {
|
||||
"collapsed": true
|
||||
},
|
||||
"outputs": [],
|
||||
"source": [
|
||||
"# Create model\n",
|
||||
"\n",
|
||||
"# Set model weights\n",
|
||||
"W = tf.Variable(tf.zeros([784, 10]))\n",
|
||||
"b = tf.Variable(tf.zeros([10]))"
|
||||
]
|
||||
},
|
||||
{
|
||||
"cell_type": "code",
|
||||
"execution_count": 10,
|
||||
"metadata": {
|
||||
"collapsed": true
|
||||
},
|
||||
"outputs": [],
|
||||
"source": [
|
||||
"# Construct model\n",
|
||||
"activation = tf.nn.softmax(tf.matmul(x, W) + b) # Softmax"
|
||||
]
|
||||
},
|
||||
{
|
||||
"cell_type": "code",
|
||||
"execution_count": 11,
|
||||
"metadata": {
|
||||
"collapsed": true
|
||||
},
|
||||
"outputs": [],
|
||||
"source": [
|
||||
"# Minimize error using cross entropy\n",
|
||||
"# Cross entropy\n",
|
||||
"cost = -tf.reduce_sum(y*tf.log(activation)) \n",
|
||||
"# Gradient Descent\n",
|
||||
"optimizer = tf.train.GradientDescentOptimizer(learning_rate).minimize(cost) "
|
||||
]
|
||||
},
|
||||
{
|
||||
"cell_type": "code",
|
||||
"execution_count": 12,
|
||||
"metadata": {
|
||||
"collapsed": true
|
||||
},
|
||||
"outputs": [],
|
||||
"source": [
|
||||
"# Initializing the variables\n",
|
||||
"init = tf.initialize_all_variables()"
|
||||
]
|
||||
},
|
||||
{
|
||||
"cell_type": "code",
|
||||
"execution_count": 13,
|
||||
"metadata": {
|
||||
"collapsed": false
|
||||
},
|
||||
"outputs": [
|
||||
{
|
||||
"name": "stdout",
|
||||
"output_type": "stream",
|
||||
"text": [
|
||||
"Epoch: 0001 cost= 29.860479714\n",
|
||||
"Epoch: 0002 cost= 22.080549484\n",
|
||||
"Epoch: 0003 cost= 21.237104595\n",
|
||||
"Epoch: 0004 cost= 20.460196280\n",
|
||||
"Epoch: 0005 cost= 20.185128237\n",
|
||||
"Epoch: 0006 cost= 19.940297202\n",
|
||||
"Epoch: 0007 cost= 19.645111119\n",
|
||||
"Epoch: 0008 cost= 19.507218031\n",
|
||||
"Epoch: 0009 cost= 19.389794492\n",
|
||||
"Epoch: 0010 cost= 19.177005816\n",
|
||||
"Epoch: 0011 cost= 19.082493615\n",
|
||||
"Epoch: 0012 cost= 19.072873598\n",
|
||||
"Epoch: 0013 cost= 18.938005402\n",
|
||||
"Epoch: 0014 cost= 18.891806430\n",
|
||||
"Epoch: 0015 cost= 18.839480221\n",
|
||||
"Epoch: 0016 cost= 18.769349510\n",
|
||||
"Epoch: 0017 cost= 18.590865587\n",
|
||||
"Epoch: 0018 cost= 18.623413677\n",
|
||||
"Epoch: 0019 cost= 18.546149085\n",
|
||||
"Epoch: 0020 cost= 18.432274895\n",
|
||||
"Epoch: 0021 cost= 18.358189004\n",
|
||||
"Epoch: 0022 cost= 18.380014628\n",
|
||||
"Epoch: 0023 cost= 18.499993471\n",
|
||||
"Epoch: 0024 cost= 18.386477311\n",
|
||||
"Epoch: 0025 cost= 18.258080609\n",
|
||||
"Optimization Finished!\n",
|
||||
"Accuracy: 0.9048\n"
|
||||
]
|
||||
}
|
||||
],
|
||||
"source": [
|
||||
"# Launch the graph\n",
|
||||
"with tf.Session() as sess:\n",
|
||||
" sess.run(init)\n",
|
||||
"\n",
|
||||
" # Training cycle\n",
|
||||
" for epoch in range(training_epochs):\n",
|
||||
" avg_cost = 0.\n",
|
||||
" total_batch = int(mnist.train.num_examples/batch_size)\n",
|
||||
" # Loop over all batches\n",
|
||||
" for i in range(total_batch):\n",
|
||||
" batch_xs, batch_ys = mnist.train.next_batch(batch_size)\n",
|
||||
" # Fit training using batch data\n",
|
||||
" sess.run(optimizer, feed_dict={x: batch_xs, y: batch_ys})\n",
|
||||
" # Compute average loss\n",
|
||||
" avg_cost += sess.run(cost, feed_dict={x: batch_xs, y: batch_ys})/total_batch\n",
|
||||
" # Display logs per epoch step\n",
|
||||
" if epoch % display_step == 0:\n",
|
||||
" print \"Epoch:\", '%04d' % (epoch+1), \"cost=\", \"{:.9f}\".format(avg_cost)\n",
|
||||
"\n",
|
||||
" print \"Optimization Finished!\"\n",
|
||||
"\n",
|
||||
" # Test model\n",
|
||||
" correct_prediction = tf.equal(tf.argmax(activation, 1), tf.argmax(y, 1))\n",
|
||||
" # Calculate accuracy\n",
|
||||
" accuracy = tf.reduce_mean(tf.cast(correct_prediction, \"float\"))\n",
|
||||
" print \"Accuracy:\", accuracy.eval({x: mnist.test.images, y: mnist.test.labels})"
|
||||
]
|
||||
}
|
||||
],
|
||||
"metadata": {
|
||||
"kernelspec": {
|
||||
"display_name": "Python 3",
|
||||
"language": "python",
|
||||
"name": "python3"
|
||||
},
|
||||
"language_info": {
|
||||
"codemirror_mode": {
|
||||
"name": "ipython",
|
||||
"version": 3
|
||||
},
|
||||
"file_extension": ".py",
|
||||
"mimetype": "text/x-python",
|
||||
"name": "python",
|
||||
"nbconvert_exporter": "python",
|
||||
"pygments_lexer": "ipython3",
|
||||
"version": "3.4.3"
|
||||
}
|
||||
},
|
||||
"nbformat": 4,
|
||||
"nbformat_minor": 0
|
||||
}
|
||||
|
|
@ -0,0 +1,386 @@
|
|||
{
|
||||
"cells": [
|
||||
{
|
||||
"cell_type": "markdown",
|
||||
"metadata": {},
|
||||
"source": [
|
||||
"# Nearest Neighbor in TensorFlow\n",
|
||||
"\n",
|
||||
"Credits: Forked from [TensorFlow-Examples](https://github.com/aymericdamien/TensorFlow-Examples) by Aymeric Damien\n",
|
||||
"\n",
|
||||
"## Setup\n",
|
||||
"\n",
|
||||
"Refer to the [setup instructions](http://nbviewer.ipython.org/github/donnemartin/data-science-ipython-notebooks/blob/master/deep-learning/tensor-flow-examples/Setup_TensorFlow.md)"
|
||||
]
|
||||
},
|
||||
{
|
||||
"cell_type": "code",
|
||||
"execution_count": 2,
|
||||
"metadata": {
|
||||
"collapsed": true
|
||||
},
|
||||
"outputs": [],
|
||||
"source": [
|
||||
"import numpy as np\n",
|
||||
"import tensorflow as tf"
|
||||
]
|
||||
},
|
||||
{
|
||||
"cell_type": "code",
|
||||
"execution_count": 3,
|
||||
"metadata": {
|
||||
"collapsed": false
|
||||
},
|
||||
"outputs": [
|
||||
{
|
||||
"name": "stdout",
|
||||
"output_type": "stream",
|
||||
"text": [
|
||||
"Extracting /tmp/data/train-images-idx3-ubyte.gz\n",
|
||||
"Extracting /tmp/data/train-labels-idx1-ubyte.gz\n",
|
||||
"Extracting /tmp/data/t10k-images-idx3-ubyte.gz\n",
|
||||
"Extracting /tmp/data/t10k-labels-idx1-ubyte.gz\n"
|
||||
]
|
||||
}
|
||||
],
|
||||
"source": [
|
||||
"# Import MINST data\n",
|
||||
"import input_data\n",
|
||||
"mnist = input_data.read_data_sets(\"/tmp/data/\", one_hot=True)"
|
||||
]
|
||||
},
|
||||
{
|
||||
"cell_type": "code",
|
||||
"execution_count": 4,
|
||||
"metadata": {
|
||||
"collapsed": true
|
||||
},
|
||||
"outputs": [],
|
||||
"source": [
|
||||
"# In this example, we limit mnist data\n",
|
||||
"Xtr, Ytr = mnist.train.next_batch(5000) #5000 for training (nn candidates)\n",
|
||||
"Xte, Yte = mnist.test.next_batch(200) #200 for testing"
|
||||
]
|
||||
},
|
||||
{
|
||||
"cell_type": "code",
|
||||
"execution_count": 5,
|
||||
"metadata": {
|
||||
"collapsed": true
|
||||
},
|
||||
"outputs": [],
|
||||
"source": [
|
||||
"# Reshape images to 1D\n",
|
||||
"Xtr = np.reshape(Xtr, newshape=(-1, 28*28))\n",
|
||||
"Xte = np.reshape(Xte, newshape=(-1, 28*28))"
|
||||
]
|
||||
},
|
||||
{
|
||||
"cell_type": "code",
|
||||
"execution_count": 6,
|
||||
"metadata": {
|
||||
"collapsed": true
|
||||
},
|
||||
"outputs": [],
|
||||
"source": [
|
||||
"# tf Graph Input\n",
|
||||
"xtr = tf.placeholder(\"float\", [None, 784])\n",
|
||||
"xte = tf.placeholder(\"float\", [784])"
|
||||
]
|
||||
},
|
||||
{
|
||||
"cell_type": "code",
|
||||
"execution_count": 8,
|
||||
"metadata": {
|
||||
"collapsed": true
|
||||
},
|
||||
"outputs": [],
|
||||
"source": [
|
||||
"# Nearest Neighbor calculation using L1 Distance\n",
|
||||
"# Calculate L1 Distance\n",
|
||||
"distance = tf.reduce_sum(tf.abs(tf.add(xtr, tf.neg(xte))), reduction_indices=1)\n",
|
||||
"# Predict: Get min distance index (Nearest neighbor)\n",
|
||||
"pred = tf.arg_min(distance, 0)\n",
|
||||
"\n",
|
||||
"accuracy = 0."
|
||||
]
|
||||
},
|
||||
{
|
||||
"cell_type": "code",
|
||||
"execution_count": 9,
|
||||
"metadata": {
|
||||
"collapsed": true
|
||||
},
|
||||
"outputs": [],
|
||||
"source": [
|
||||
"# Initializing the variables\n",
|
||||
"init = tf.initialize_all_variables()"
|
||||
]
|
||||
},
|
||||
{
|
||||
"cell_type": "code",
|
||||
"execution_count": 10,
|
||||
"metadata": {
|
||||
"collapsed": false
|
||||
},
|
||||
"outputs": [
|
||||
{
|
||||
"name": "stdout",
|
||||
"output_type": "stream",
|
||||
"text": [
|
||||
"Test 0 Prediction: 7 True Class: 7\n",
|
||||
"Test 1 Prediction: 2 True Class: 2\n",
|
||||
"Test 2 Prediction: 1 True Class: 1\n",
|
||||
"Test 3 Prediction: 0 True Class: 0\n",
|
||||
"Test 4 Prediction: 4 True Class: 4\n",
|
||||
"Test 5 Prediction: 1 True Class: 1\n",
|
||||
"Test 6 Prediction: 4 True Class: 4\n",
|
||||
"Test 7 Prediction: 9 True Class: 9\n",
|
||||
"Test 8 Prediction: 8 True Class: 5\n",
|
||||
"Test 9 Prediction: 9 True Class: 9\n",
|
||||
"Test 10 Prediction: 0 True Class: 0\n",
|
||||
"Test 11 Prediction: 0 True Class: 6\n",
|
||||
"Test 12 Prediction: 9 True Class: 9\n",
|
||||
"Test 13 Prediction: 0 True Class: 0\n",
|
||||
"Test 14 Prediction: 1 True Class: 1\n",
|
||||
"Test 15 Prediction: 5 True Class: 5\n",
|
||||
"Test 16 Prediction: 4 True Class: 9\n",
|
||||
"Test 17 Prediction: 7 True Class: 7\n",
|
||||
"Test 18 Prediction: 3 True Class: 3\n",
|
||||
"Test 19 Prediction: 4 True Class: 4\n",
|
||||
"Test 20 Prediction: 9 True Class: 9\n",
|
||||
"Test 21 Prediction: 6 True Class: 6\n",
|
||||
"Test 22 Prediction: 6 True Class: 6\n",
|
||||
"Test 23 Prediction: 5 True Class: 5\n",
|
||||
"Test 24 Prediction: 4 True Class: 4\n",
|
||||
"Test 25 Prediction: 0 True Class: 0\n",
|
||||
"Test 26 Prediction: 7 True Class: 7\n",
|
||||
"Test 27 Prediction: 4 True Class: 4\n",
|
||||
"Test 28 Prediction: 0 True Class: 0\n",
|
||||
"Test 29 Prediction: 1 True Class: 1\n",
|
||||
"Test 30 Prediction: 3 True Class: 3\n",
|
||||
"Test 31 Prediction: 1 True Class: 1\n",
|
||||
"Test 32 Prediction: 3 True Class: 3\n",
|
||||
"Test 33 Prediction: 4 True Class: 4\n",
|
||||
"Test 34 Prediction: 7 True Class: 7\n",
|
||||
"Test 35 Prediction: 2 True Class: 2\n",
|
||||
"Test 36 Prediction: 7 True Class: 7\n",
|
||||
"Test 37 Prediction: 1 True Class: 1\n",
|
||||
"Test 38 Prediction: 2 True Class: 2\n",
|
||||
"Test 39 Prediction: 1 True Class: 1\n",
|
||||
"Test 40 Prediction: 1 True Class: 1\n",
|
||||
"Test 41 Prediction: 7 True Class: 7\n",
|
||||
"Test 42 Prediction: 4 True Class: 4\n",
|
||||
"Test 43 Prediction: 1 True Class: 2\n",
|
||||
"Test 44 Prediction: 3 True Class: 3\n",
|
||||
"Test 45 Prediction: 5 True Class: 5\n",
|
||||
"Test 46 Prediction: 1 True Class: 1\n",
|
||||
"Test 47 Prediction: 2 True Class: 2\n",
|
||||
"Test 48 Prediction: 4 True Class: 4\n",
|
||||
"Test 49 Prediction: 4 True Class: 4\n",
|
||||
"Test 50 Prediction: 6 True Class: 6\n",
|
||||
"Test 51 Prediction: 3 True Class: 3\n",
|
||||
"Test 52 Prediction: 5 True Class: 5\n",
|
||||
"Test 53 Prediction: 5 True Class: 5\n",
|
||||
"Test 54 Prediction: 6 True Class: 6\n",
|
||||
"Test 55 Prediction: 0 True Class: 0\n",
|
||||
"Test 56 Prediction: 4 True Class: 4\n",
|
||||
"Test 57 Prediction: 1 True Class: 1\n",
|
||||
"Test 58 Prediction: 9 True Class: 9\n",
|
||||
"Test 59 Prediction: 5 True Class: 5\n",
|
||||
"Test 60 Prediction: 7 True Class: 7\n",
|
||||
"Test 61 Prediction: 8 True Class: 8\n",
|
||||
"Test 62 Prediction: 9 True Class: 9\n",
|
||||
"Test 63 Prediction: 3 True Class: 3\n",
|
||||
"Test 64 Prediction: 7 True Class: 7\n",
|
||||
"Test 65 Prediction: 4 True Class: 4\n",
|
||||
"Test 66 Prediction: 6 True Class: 6\n",
|
||||
"Test 67 Prediction: 4 True Class: 4\n",
|
||||
"Test 68 Prediction: 3 True Class: 3\n",
|
||||
"Test 69 Prediction: 0 True Class: 0\n",
|
||||
"Test 70 Prediction: 7 True Class: 7\n",
|
||||
"Test 71 Prediction: 0 True Class: 0\n",
|
||||
"Test 72 Prediction: 2 True Class: 2\n",
|
||||
"Test 73 Prediction: 7 True Class: 9\n",
|
||||
"Test 74 Prediction: 1 True Class: 1\n",
|
||||
"Test 75 Prediction: 7 True Class: 7\n",
|
||||
"Test 76 Prediction: 3 True Class: 3\n",
|
||||
"Test 77 Prediction: 7 True Class: 2\n",
|
||||
"Test 78 Prediction: 9 True Class: 9\n",
|
||||
"Test 79 Prediction: 7 True Class: 7\n",
|
||||
"Test 80 Prediction: 7 True Class: 7\n",
|
||||
"Test 81 Prediction: 6 True Class: 6\n",
|
||||
"Test 82 Prediction: 2 True Class: 2\n",
|
||||
"Test 83 Prediction: 7 True Class: 7\n",
|
||||
"Test 84 Prediction: 8 True Class: 8\n",
|
||||
"Test 85 Prediction: 4 True Class: 4\n",
|
||||
"Test 86 Prediction: 7 True Class: 7\n",
|
||||
"Test 87 Prediction: 3 True Class: 3\n",
|
||||
"Test 88 Prediction: 6 True Class: 6\n",
|
||||
"Test 89 Prediction: 1 True Class: 1\n",
|
||||
"Test 90 Prediction: 3 True Class: 3\n",
|
||||
"Test 91 Prediction: 6 True Class: 6\n",
|
||||
"Test 92 Prediction: 9 True Class: 9\n",
|
||||
"Test 93 Prediction: 3 True Class: 3\n",
|
||||
"Test 94 Prediction: 1 True Class: 1\n",
|
||||
"Test 95 Prediction: 4 True Class: 4\n",
|
||||
"Test 96 Prediction: 1 True Class: 1\n",
|
||||
"Test 97 Prediction: 7 True Class: 7\n",
|
||||
"Test 98 Prediction: 6 True Class: 6\n",
|
||||
"Test 99 Prediction: 9 True Class: 9\n",
|
||||
"Test 100 Prediction: 6 True Class: 6\n",
|
||||
"Test 101 Prediction: 0 True Class: 0\n",
|
||||
"Test 102 Prediction: 5 True Class: 5\n",
|
||||
"Test 103 Prediction: 4 True Class: 4\n",
|
||||
"Test 104 Prediction: 9 True Class: 9\n",
|
||||
"Test 105 Prediction: 9 True Class: 9\n",
|
||||
"Test 106 Prediction: 2 True Class: 2\n",
|
||||
"Test 107 Prediction: 1 True Class: 1\n",
|
||||
"Test 108 Prediction: 9 True Class: 9\n",
|
||||
"Test 109 Prediction: 4 True Class: 4\n",
|
||||
"Test 110 Prediction: 8 True Class: 8\n",
|
||||
"Test 111 Prediction: 7 True Class: 7\n",
|
||||
"Test 112 Prediction: 3 True Class: 3\n",
|
||||
"Test 113 Prediction: 9 True Class: 9\n",
|
||||
"Test 114 Prediction: 7 True Class: 7\n",
|
||||
"Test 115 Prediction: 9 True Class: 4\n",
|
||||
"Test 116 Prediction: 9 True Class: 4\n",
|
||||
"Test 117 Prediction: 4 True Class: 4\n",
|
||||
"Test 118 Prediction: 9 True Class: 9\n",
|
||||
"Test 119 Prediction: 7 True Class: 2\n",
|
||||
"Test 120 Prediction: 5 True Class: 5\n",
|
||||
"Test 121 Prediction: 4 True Class: 4\n",
|
||||
"Test 122 Prediction: 7 True Class: 7\n",
|
||||
"Test 123 Prediction: 6 True Class: 6\n",
|
||||
"Test 124 Prediction: 7 True Class: 7\n",
|
||||
"Test 125 Prediction: 9 True Class: 9\n",
|
||||
"Test 126 Prediction: 0 True Class: 0\n",
|
||||
"Test 127 Prediction: 5 True Class: 5\n",
|
||||
"Test 128 Prediction: 8 True Class: 8\n",
|
||||
"Test 129 Prediction: 5 True Class: 5\n",
|
||||
"Test 130 Prediction: 6 True Class: 6\n",
|
||||
"Test 131 Prediction: 6 True Class: 6\n",
|
||||
"Test 132 Prediction: 5 True Class: 5\n",
|
||||
"Test 133 Prediction: 7 True Class: 7\n",
|
||||
"Test 134 Prediction: 8 True Class: 8\n",
|
||||
"Test 135 Prediction: 1 True Class: 1\n",
|
||||
"Test 136 Prediction: 0 True Class: 0\n",
|
||||
"Test 137 Prediction: 1 True Class: 1\n",
|
||||
"Test 138 Prediction: 6 True Class: 6\n",
|
||||
"Test 139 Prediction: 4 True Class: 4\n",
|
||||
"Test 140 Prediction: 6 True Class: 6\n",
|
||||
"Test 141 Prediction: 7 True Class: 7\n",
|
||||
"Test 142 Prediction: 2 True Class: 3\n",
|
||||
"Test 143 Prediction: 1 True Class: 1\n",
|
||||
"Test 144 Prediction: 7 True Class: 7\n",
|
||||
"Test 145 Prediction: 1 True Class: 1\n",
|
||||
"Test 146 Prediction: 8 True Class: 8\n",
|
||||
"Test 147 Prediction: 2 True Class: 2\n",
|
||||
"Test 148 Prediction: 0 True Class: 0\n",
|
||||
"Test 149 Prediction: 1 True Class: 2\n",
|
||||
"Test 150 Prediction: 9 True Class: 9\n",
|
||||
"Test 151 Prediction: 9 True Class: 9\n",
|
||||
"Test 152 Prediction: 5 True Class: 5\n",
|
||||
"Test 153 Prediction: 5 True Class: 5\n",
|
||||
"Test 154 Prediction: 1 True Class: 1\n",
|
||||
"Test 155 Prediction: 5 True Class: 5\n",
|
||||
"Test 156 Prediction: 6 True Class: 6\n",
|
||||
"Test 157 Prediction: 0 True Class: 0\n",
|
||||
"Test 158 Prediction: 3 True Class: 3\n",
|
||||
"Test 159 Prediction: 4 True Class: 4\n",
|
||||
"Test 160 Prediction: 4 True Class: 4\n",
|
||||
"Test 161 Prediction: 6 True Class: 6\n",
|
||||
"Test 162 Prediction: 5 True Class: 5\n",
|
||||
"Test 163 Prediction: 4 True Class: 4\n",
|
||||
"Test 164 Prediction: 6 True Class: 6\n",
|
||||
"Test 165 Prediction: 5 True Class: 5\n",
|
||||
"Test 166 Prediction: 4 True Class: 4\n",
|
||||
"Test 167 Prediction: 5 True Class: 5\n",
|
||||
"Test 168 Prediction: 1 True Class: 1\n",
|
||||
"Test 169 Prediction: 4 True Class: 4\n",
|
||||
"Test 170 Prediction: 9 True Class: 4\n",
|
||||
"Test 171 Prediction: 7 True Class: 7\n",
|
||||
"Test 172 Prediction: 2 True Class: 2\n",
|
||||
"Test 173 Prediction: 3 True Class: 3\n",
|
||||
"Test 174 Prediction: 2 True Class: 2\n",
|
||||
"Test 175 Prediction: 1 True Class: 7\n",
|
||||
"Test 176 Prediction: 1 True Class: 1\n",
|
||||
"Test 177 Prediction: 8 True Class: 8\n",
|
||||
"Test 178 Prediction: 1 True Class: 1\n",
|
||||
"Test 179 Prediction: 8 True Class: 8\n",
|
||||
"Test 180 Prediction: 1 True Class: 1\n",
|
||||
"Test 181 Prediction: 8 True Class: 8\n",
|
||||
"Test 182 Prediction: 5 True Class: 5\n",
|
||||
"Test 183 Prediction: 0 True Class: 0\n",
|
||||
"Test 184 Prediction: 2 True Class: 8\n",
|
||||
"Test 185 Prediction: 9 True Class: 9\n",
|
||||
"Test 186 Prediction: 2 True Class: 2\n",
|
||||
"Test 187 Prediction: 5 True Class: 5\n",
|
||||
"Test 188 Prediction: 0 True Class: 0\n",
|
||||
"Test 189 Prediction: 1 True Class: 1\n",
|
||||
"Test 190 Prediction: 1 True Class: 1\n",
|
||||
"Test 191 Prediction: 1 True Class: 1\n",
|
||||
"Test 192 Prediction: 0 True Class: 0\n",
|
||||
"Test 193 Prediction: 4 True Class: 9\n",
|
||||
"Test 194 Prediction: 0 True Class: 0\n",
|
||||
"Test 195 Prediction: 1 True Class: 3\n",
|
||||
"Test 196 Prediction: 1 True Class: 1\n",
|
||||
"Test 197 Prediction: 6 True Class: 6\n",
|
||||
"Test 198 Prediction: 4 True Class: 4\n",
|
||||
"Test 199 Prediction: 2 True Class: 2\n",
|
||||
"Done!\n",
|
||||
"Accuracy: 0.92\n"
|
||||
]
|
||||
}
|
||||
],
|
||||
"source": [
|
||||
"# Launch the graph\n",
|
||||
"with tf.Session() as sess:\n",
|
||||
" sess.run(init)\n",
|
||||
"\n",
|
||||
" # loop over test data\n",
|
||||
" for i in range(len(Xte)):\n",
|
||||
" # Get nearest neighbor\n",
|
||||
" nn_index = sess.run(pred, feed_dict={xtr: Xtr, xte: Xte[i,:]})\n",
|
||||
" # Get nearest neighbor class label and compare it to its true label\n",
|
||||
" print \"Test\", i, \"Prediction:\", np.argmax(Ytr[nn_index]), \\\n",
|
||||
" \"True Class:\", np.argmax(Yte[i])\n",
|
||||
" # Calculate accuracy\n",
|
||||
" if np.argmax(Ytr[nn_index]) == np.argmax(Yte[i]):\n",
|
||||
" accuracy += 1./len(Xte)\n",
|
||||
" print \"Done!\"\n",
|
||||
" print \"Accuracy:\", accuracy"
|
||||
]
|
||||
},
|
||||
{
|
||||
"cell_type": "code",
|
||||
"execution_count": null,
|
||||
"metadata": {
|
||||
"collapsed": true
|
||||
},
|
||||
"outputs": [],
|
||||
"source": []
|
||||
}
|
||||
],
|
||||
"metadata": {
|
||||
"kernelspec": {
|
||||
"display_name": "Python 3",
|
||||
"language": "python",
|
||||
"name": "python3"
|
||||
},
|
||||
"language_info": {
|
||||
"codemirror_mode": {
|
||||
"name": "ipython",
|
||||
"version": 3
|
||||
},
|
||||
"file_extension": ".py",
|
||||
"mimetype": "text/x-python",
|
||||
"name": "python",
|
||||
"nbconvert_exporter": "python",
|
||||
"pygments_lexer": "ipython3",
|
||||
"version": "3.4.3"
|
||||
}
|
||||
},
|
||||
"nbformat": 4,
|
||||
"nbformat_minor": 0
|
||||
}
|
||||
|
|
@ -0,0 +1,345 @@
|
|||
{
|
||||
"cells": [
|
||||
{
|
||||
"cell_type": "markdown",
|
||||
"metadata": {},
|
||||
"source": [
|
||||
"# AlexNet in TensorFlow\n",
|
||||
"\n",
|
||||
"Credits: Forked from [TensorFlow-Examples](https://github.com/aymericdamien/TensorFlow-Examples) by Aymeric Damien\n",
|
||||
"\n",
|
||||
"## Setup\n",
|
||||
"\n",
|
||||
"Refer to the [setup instructions](http://nbviewer.ipython.org/github/donnemartin/data-science-ipython-notebooks/blob/master/deep-learning/tensor-flow-examples/Setup_TensorFlow.md)"
|
||||
]
|
||||
},
|
||||
{
|
||||
"cell_type": "code",
|
||||
"execution_count": 2,
|
||||
"metadata": {
|
||||
"collapsed": false
|
||||
},
|
||||
"outputs": [
|
||||
{
|
||||
"name": "stdout",
|
||||
"output_type": "stream",
|
||||
"text": [
|
||||
"Extracting /tmp/data/train-images-idx3-ubyte.gz\n",
|
||||
"Extracting /tmp/data/train-labels-idx1-ubyte.gz\n",
|
||||
"Extracting /tmp/data/t10k-images-idx3-ubyte.gz\n",
|
||||
"Extracting /tmp/data/t10k-labels-idx1-ubyte.gz\n"
|
||||
]
|
||||
}
|
||||
],
|
||||
"source": [
|
||||
"# Import MINST data\n",
|
||||
"import input_data\n",
|
||||
"mnist = input_data.read_data_sets(\"/tmp/data/\", one_hot=True)"
|
||||
]
|
||||
},
|
||||
{
|
||||
"cell_type": "code",
|
||||
"execution_count": 3,
|
||||
"metadata": {
|
||||
"collapsed": true
|
||||
},
|
||||
"outputs": [],
|
||||
"source": [
|
||||
"import tensorflow as tf"
|
||||
]
|
||||
},
|
||||
{
|
||||
"cell_type": "code",
|
||||
"execution_count": 17,
|
||||
"metadata": {
|
||||
"collapsed": true
|
||||
},
|
||||
"outputs": [],
|
||||
"source": [
|
||||
"# Parameters\n",
|
||||
"learning_rate = 0.001\n",
|
||||
"training_iters = 300000\n",
|
||||
"batch_size = 64\n",
|
||||
"display_step = 100"
|
||||
]
|
||||
},
|
||||
{
|
||||
"cell_type": "code",
|
||||
"execution_count": 5,
|
||||
"metadata": {
|
||||
"collapsed": true
|
||||
},
|
||||
"outputs": [],
|
||||
"source": [
|
||||
"# Network Parameters\n",
|
||||
"n_input = 784 # MNIST data input (img shape: 28*28)\n",
|
||||
"n_classes = 10 # MNIST total classes (0-9 digits)\n",
|
||||
"dropout = 0.8 # Dropout, probability to keep units"
|
||||
]
|
||||
},
|
||||
{
|
||||
"cell_type": "code",
|
||||
"execution_count": 6,
|
||||
"metadata": {
|
||||
"collapsed": true
|
||||
},
|
||||
"outputs": [],
|
||||
"source": [
|
||||
"# tf Graph input\n",
|
||||
"x = tf.placeholder(tf.types.float32, [None, n_input])\n",
|
||||
"y = tf.placeholder(tf.types.float32, [None, n_classes])\n",
|
||||
"keep_prob = tf.placeholder(tf.types.float32) # dropout (keep probability)"
|
||||
]
|
||||
},
|
||||
{
|
||||
"cell_type": "code",
|
||||
"execution_count": 7,
|
||||
"metadata": {
|
||||
"collapsed": true
|
||||
},
|
||||
"outputs": [],
|
||||
"source": [
|
||||
"# Create AlexNet model\n",
|
||||
"def conv2d(name, l_input, w, b):\n",
|
||||
" return tf.nn.relu(tf.nn.bias_add(tf.nn.conv2d(l_input, w, strides=[1, 1, 1, 1], \n",
|
||||
" padding='SAME'),b), name=name)\n",
|
||||
"\n",
|
||||
"def max_pool(name, l_input, k):\n",
|
||||
" return tf.nn.max_pool(l_input, ksize=[1, k, k, 1], strides=[1, k, k, 1], \n",
|
||||
" padding='SAME', name=name)\n",
|
||||
"\n",
|
||||
"def norm(name, l_input, lsize=4):\n",
|
||||
" return tf.nn.lrn(l_input, lsize, bias=1.0, alpha=0.001 / 9.0, beta=0.75, name=name)\n",
|
||||
"\n",
|
||||
"def alex_net(_X, _weights, _biases, _dropout):\n",
|
||||
" # Reshape input picture\n",
|
||||
" _X = tf.reshape(_X, shape=[-1, 28, 28, 1])\n",
|
||||
"\n",
|
||||
" # Convolution Layer\n",
|
||||
" conv1 = conv2d('conv1', _X, _weights['wc1'], _biases['bc1'])\n",
|
||||
" # Max Pooling (down-sampling)\n",
|
||||
" pool1 = max_pool('pool1', conv1, k=2)\n",
|
||||
" # Apply Normalization\n",
|
||||
" norm1 = norm('norm1', pool1, lsize=4)\n",
|
||||
" # Apply Dropout\n",
|
||||
" norm1 = tf.nn.dropout(norm1, _dropout)\n",
|
||||
"\n",
|
||||
" # Convolution Layer\n",
|
||||
" conv2 = conv2d('conv2', norm1, _weights['wc2'], _biases['bc2'])\n",
|
||||
" # Max Pooling (down-sampling)\n",
|
||||
" pool2 = max_pool('pool2', conv2, k=2)\n",
|
||||
" # Apply Normalization\n",
|
||||
" norm2 = norm('norm2', pool2, lsize=4)\n",
|
||||
" # Apply Dropout\n",
|
||||
" norm2 = tf.nn.dropout(norm2, _dropout)\n",
|
||||
"\n",
|
||||
" # Convolution Layer\n",
|
||||
" conv3 = conv2d('conv3', norm2, _weights['wc3'], _biases['bc3'])\n",
|
||||
" # Max Pooling (down-sampling)\n",
|
||||
" pool3 = max_pool('pool3', conv3, k=2)\n",
|
||||
" # Apply Normalization\n",
|
||||
" norm3 = norm('norm3', pool3, lsize=4)\n",
|
||||
" # Apply Dropout\n",
|
||||
" norm3 = tf.nn.dropout(norm3, _dropout)\n",
|
||||
"\n",
|
||||
" # Fully connected layer\n",
|
||||
" # Reshape conv3 output to fit dense layer input\n",
|
||||
" dense1 = tf.reshape(norm3, [-1, _weights['wd1'].get_shape().as_list()[0]]) \n",
|
||||
" # Relu activation\n",
|
||||
" dense1 = tf.nn.relu(tf.matmul(dense1, _weights['wd1']) + _biases['bd1'], name='fc1')\n",
|
||||
" \n",
|
||||
" # Relu activation\n",
|
||||
" dense2 = tf.nn.relu(tf.matmul(dense1, _weights['wd2']) + _biases['bd2'], name='fc2') \n",
|
||||
"\n",
|
||||
" # Output, class prediction\n",
|
||||
" out = tf.matmul(dense2, _weights['out']) + _biases['out']\n",
|
||||
" return out"
|
||||
]
|
||||
},
|
||||
{
|
||||
"cell_type": "code",
|
||||
"execution_count": 8,
|
||||
"metadata": {
|
||||
"collapsed": true
|
||||
},
|
||||
"outputs": [],
|
||||
"source": [
|
||||
"# Store layers weight & bias\n",
|
||||
"weights = {\n",
|
||||
" 'wc1': tf.Variable(tf.random_normal([3, 3, 1, 64])),\n",
|
||||
" 'wc2': tf.Variable(tf.random_normal([3, 3, 64, 128])),\n",
|
||||
" 'wc3': tf.Variable(tf.random_normal([3, 3, 128, 256])),\n",
|
||||
" 'wd1': tf.Variable(tf.random_normal([4*4*256, 1024])),\n",
|
||||
" 'wd2': tf.Variable(tf.random_normal([1024, 1024])),\n",
|
||||
" 'out': tf.Variable(tf.random_normal([1024, 10]))\n",
|
||||
"}\n",
|
||||
"biases = {\n",
|
||||
" 'bc1': tf.Variable(tf.random_normal([64])),\n",
|
||||
" 'bc2': tf.Variable(tf.random_normal([128])),\n",
|
||||
" 'bc3': tf.Variable(tf.random_normal([256])),\n",
|
||||
" 'bd1': tf.Variable(tf.random_normal([1024])),\n",
|
||||
" 'bd2': tf.Variable(tf.random_normal([1024])),\n",
|
||||
" 'out': tf.Variable(tf.random_normal([n_classes]))\n",
|
||||
"}"
|
||||
]
|
||||
},
|
||||
{
|
||||
"cell_type": "code",
|
||||
"execution_count": 9,
|
||||
"metadata": {
|
||||
"collapsed": true
|
||||
},
|
||||
"outputs": [],
|
||||
"source": [
|
||||
"# Construct model\n",
|
||||
"pred = alex_net(x, weights, biases, keep_prob)"
|
||||
]
|
||||
},
|
||||
{
|
||||
"cell_type": "code",
|
||||
"execution_count": 10,
|
||||
"metadata": {
|
||||
"collapsed": true
|
||||
},
|
||||
"outputs": [],
|
||||
"source": [
|
||||
"# Define loss and optimizer\n",
|
||||
"cost = tf.reduce_mean(tf.nn.softmax_cross_entropy_with_logits(pred, y))\n",
|
||||
"optimizer = tf.train.AdamOptimizer(learning_rate=learning_rate).minimize(cost)"
|
||||
]
|
||||
},
|
||||
{
|
||||
"cell_type": "code",
|
||||
"execution_count": 11,
|
||||
"metadata": {
|
||||
"collapsed": true
|
||||
},
|
||||
"outputs": [],
|
||||
"source": [
|
||||
"# Evaluate model\n",
|
||||
"correct_pred = tf.equal(tf.argmax(pred,1), tf.argmax(y,1))\n",
|
||||
"accuracy = tf.reduce_mean(tf.cast(correct_pred, tf.types.float32))"
|
||||
]
|
||||
},
|
||||
{
|
||||
"cell_type": "code",
|
||||
"execution_count": 12,
|
||||
"metadata": {
|
||||
"collapsed": true
|
||||
},
|
||||
"outputs": [],
|
||||
"source": [
|
||||
"# Initializing the variables\n",
|
||||
"init = tf.initialize_all_variables()"
|
||||
]
|
||||
},
|
||||
{
|
||||
"cell_type": "code",
|
||||
"execution_count": 18,
|
||||
"metadata": {
|
||||
"collapsed": false
|
||||
},
|
||||
"outputs": [
|
||||
{
|
||||
"name": "stdout",
|
||||
"output_type": "stream",
|
||||
"text": [
|
||||
"Iter 6400, Minibatch Loss= 29666.185547, Training Accuracy= 0.59375\n",
|
||||
"Iter 12800, Minibatch Loss= 22125.562500, Training Accuracy= 0.60938\n",
|
||||
"Iter 19200, Minibatch Loss= 22631.134766, Training Accuracy= 0.59375\n",
|
||||
"Iter 25600, Minibatch Loss= 18498.414062, Training Accuracy= 0.62500\n",
|
||||
"Iter 32000, Minibatch Loss= 11318.283203, Training Accuracy= 0.70312\n",
|
||||
"Iter 38400, Minibatch Loss= 12076.280273, Training Accuracy= 0.70312\n",
|
||||
"Iter 44800, Minibatch Loss= 8195.520508, Training Accuracy= 0.82812\n",
|
||||
"Iter 51200, Minibatch Loss= 5176.181641, Training Accuracy= 0.84375\n",
|
||||
"Iter 57600, Minibatch Loss= 8951.896484, Training Accuracy= 0.81250\n",
|
||||
"Iter 64000, Minibatch Loss= 10096.946289, Training Accuracy= 0.78125\n",
|
||||
"Iter 70400, Minibatch Loss= 11466.641602, Training Accuracy= 0.68750\n",
|
||||
"Iter 76800, Minibatch Loss= 7469.824219, Training Accuracy= 0.78125\n",
|
||||
"Iter 83200, Minibatch Loss= 4147.449219, Training Accuracy= 0.89062\n",
|
||||
"Iter 89600, Minibatch Loss= 5904.782227, Training Accuracy= 0.82812\n",
|
||||
"Iter 96000, Minibatch Loss= 718.493713, Training Accuracy= 0.93750\n",
|
||||
"Iter 102400, Minibatch Loss= 2184.151367, Training Accuracy= 0.93750\n",
|
||||
"Iter 108800, Minibatch Loss= 2354.463135, Training Accuracy= 0.89062\n",
|
||||
"Iter 115200, Minibatch Loss= 8612.959961, Training Accuracy= 0.81250\n",
|
||||
"Iter 121600, Minibatch Loss= 2225.773926, Training Accuracy= 0.84375\n",
|
||||
"Iter 128000, Minibatch Loss= 160.583618, Training Accuracy= 0.96875\n",
|
||||
"Iter 134400, Minibatch Loss= 1524.846069, Training Accuracy= 0.93750\n",
|
||||
"Iter 140800, Minibatch Loss= 3501.871094, Training Accuracy= 0.89062\n",
|
||||
"Iter 147200, Minibatch Loss= 661.977051, Training Accuracy= 0.96875\n",
|
||||
"Iter 153600, Minibatch Loss= 367.857788, Training Accuracy= 0.98438\n",
|
||||
"Iter 160000, Minibatch Loss= 1735.458740, Training Accuracy= 0.90625\n",
|
||||
"Iter 166400, Minibatch Loss= 209.320374, Training Accuracy= 0.95312\n",
|
||||
"Iter 172800, Minibatch Loss= 1788.553955, Training Accuracy= 0.90625\n",
|
||||
"Iter 179200, Minibatch Loss= 912.995544, Training Accuracy= 0.93750\n",
|
||||
"Iter 185600, Minibatch Loss= 2534.074463, Training Accuracy= 0.87500\n",
|
||||
"Iter 192000, Minibatch Loss= 73.052612, Training Accuracy= 0.96875\n",
|
||||
"Iter 198400, Minibatch Loss= 1609.606323, Training Accuracy= 0.93750\n",
|
||||
"Iter 204800, Minibatch Loss= 1823.219727, Training Accuracy= 0.96875\n",
|
||||
"Iter 211200, Minibatch Loss= 578.051086, Training Accuracy= 0.96875\n",
|
||||
"Iter 217600, Minibatch Loss= 1532.326172, Training Accuracy= 0.89062\n",
|
||||
"Iter 224000, Minibatch Loss= 769.775269, Training Accuracy= 0.95312\n",
|
||||
"Iter 230400, Minibatch Loss= 2614.737793, Training Accuracy= 0.92188\n",
|
||||
"Iter 236800, Minibatch Loss= 938.664368, Training Accuracy= 0.95312\n",
|
||||
"Iter 243200, Minibatch Loss= 1520.495605, Training Accuracy= 0.93750\n",
|
||||
"Iter 249600, Minibatch Loss= 657.419739, Training Accuracy= 0.95312\n",
|
||||
"Iter 256000, Minibatch Loss= 522.802124, Training Accuracy= 0.90625\n",
|
||||
"Iter 262400, Minibatch Loss= 211.188477, Training Accuracy= 0.96875\n",
|
||||
"Iter 268800, Minibatch Loss= 520.451172, Training Accuracy= 0.92188\n",
|
||||
"Iter 275200, Minibatch Loss= 1418.759155, Training Accuracy= 0.89062\n",
|
||||
"Iter 281600, Minibatch Loss= 241.748596, Training Accuracy= 0.96875\n",
|
||||
"Iter 288000, Minibatch Loss= 0.000000, Training Accuracy= 1.00000\n",
|
||||
"Iter 294400, Minibatch Loss= 1535.772827, Training Accuracy= 0.92188\n",
|
||||
"Optimization Finished!\n",
|
||||
"Testing Accuracy: 0.980469\n"
|
||||
]
|
||||
}
|
||||
],
|
||||
"source": [
|
||||
"# Launch the graph\n",
|
||||
"with tf.Session() as sess:\n",
|
||||
" sess.run(init)\n",
|
||||
" step = 1\n",
|
||||
" # Keep training until reach max iterations\n",
|
||||
" while step * batch_size < training_iters:\n",
|
||||
" batch_xs, batch_ys = mnist.train.next_batch(batch_size)\n",
|
||||
" # Fit training using batch data\n",
|
||||
" sess.run(optimizer, feed_dict={x: batch_xs, y: batch_ys, keep_prob: dropout})\n",
|
||||
" if step % display_step == 0:\n",
|
||||
" # Calculate batch accuracy\n",
|
||||
" acc = sess.run(accuracy, feed_dict={x: batch_xs, y: batch_ys, keep_prob: 1.})\n",
|
||||
" # Calculate batch loss\n",
|
||||
" loss = sess.run(cost, feed_dict={x: batch_xs, y: batch_ys, keep_prob: 1.})\n",
|
||||
" print \"Iter \" + str(step*batch_size) + \", Minibatch Loss= \" \\\n",
|
||||
" + \"{:.6f}\".format(loss) + \", Training Accuracy= \" + \"{:.5f}\".format(acc)\n",
|
||||
" step += 1\n",
|
||||
" print \"Optimization Finished!\"\n",
|
||||
" # Calculate accuracy for 256 mnist test images\n",
|
||||
" print \"Testing Accuracy:\", sess.run(accuracy, feed_dict={x: mnist.test.images[:256], \n",
|
||||
" y: mnist.test.labels[:256], \n",
|
||||
" keep_prob: 1.})"
|
||||
]
|
||||
}
|
||||
],
|
||||
"metadata": {
|
||||
"kernelspec": {
|
||||
"display_name": "Python 3",
|
||||
"language": "python",
|
||||
"name": "python3"
|
||||
},
|
||||
"language_info": {
|
||||
"codemirror_mode": {
|
||||
"name": "ipython",
|
||||
"version": 3
|
||||
},
|
||||
"file_extension": ".py",
|
||||
"mimetype": "text/x-python",
|
||||
"name": "python",
|
||||
"nbconvert_exporter": "python",
|
||||
"pygments_lexer": "ipython3",
|
||||
"version": "3.4.3"
|
||||
}
|
||||
},
|
||||
"nbformat": 4,
|
||||
"nbformat_minor": 0
|
||||
}
|
||||
|
|
@ -0,0 +1,321 @@
|
|||
{
|
||||
"cells": [
|
||||
{
|
||||
"cell_type": "markdown",
|
||||
"metadata": {},
|
||||
"source": [
|
||||
"# Convolutional Neural Network in TensorFlow\n",
|
||||
"\n",
|
||||
"Credits: Forked from [TensorFlow-Examples](https://github.com/aymericdamien/TensorFlow-Examples) by Aymeric Damien\n",
|
||||
"\n",
|
||||
"## Setup\n",
|
||||
"\n",
|
||||
"Refer to the [setup instructions](http://nbviewer.ipython.org/github/donnemartin/data-science-ipython-notebooks/blob/master/deep-learning/tensor-flow-examples/Setup_TensorFlow.md)"
|
||||
]
|
||||
},
|
||||
{
|
||||
"cell_type": "code",
|
||||
"execution_count": 2,
|
||||
"metadata": {
|
||||
"collapsed": false
|
||||
},
|
||||
"outputs": [
|
||||
{
|
||||
"name": "stdout",
|
||||
"output_type": "stream",
|
||||
"text": [
|
||||
"Extracting /tmp/data/train-images-idx3-ubyte.gz\n",
|
||||
"Extracting /tmp/data/train-labels-idx1-ubyte.gz\n",
|
||||
"Extracting /tmp/data/t10k-images-idx3-ubyte.gz\n",
|
||||
"Extracting /tmp/data/t10k-labels-idx1-ubyte.gz\n"
|
||||
]
|
||||
}
|
||||
],
|
||||
"source": [
|
||||
"# Import MINST data\n",
|
||||
"import input_data\n",
|
||||
"mnist = input_data.read_data_sets(\"/tmp/data/\", one_hot=True)"
|
||||
]
|
||||
},
|
||||
{
|
||||
"cell_type": "code",
|
||||
"execution_count": 3,
|
||||
"metadata": {
|
||||
"collapsed": true
|
||||
},
|
||||
"outputs": [],
|
||||
"source": [
|
||||
"import tensorflow as tf"
|
||||
]
|
||||
},
|
||||
{
|
||||
"cell_type": "code",
|
||||
"execution_count": 18,
|
||||
"metadata": {
|
||||
"collapsed": true
|
||||
},
|
||||
"outputs": [],
|
||||
"source": [
|
||||
"# Parameters\n",
|
||||
"learning_rate = 0.001\n",
|
||||
"training_iters = 100000\n",
|
||||
"batch_size = 128\n",
|
||||
"display_step = 20"
|
||||
]
|
||||
},
|
||||
{
|
||||
"cell_type": "code",
|
||||
"execution_count": 5,
|
||||
"metadata": {
|
||||
"collapsed": true
|
||||
},
|
||||
"outputs": [],
|
||||
"source": [
|
||||
"# Network Parameters\n",
|
||||
"n_input = 784 # MNIST data input (img shape: 28*28)\n",
|
||||
"n_classes = 10 # MNIST total classes (0-9 digits)\n",
|
||||
"dropout = 0.75 # Dropout, probability to keep units"
|
||||
]
|
||||
},
|
||||
{
|
||||
"cell_type": "code",
|
||||
"execution_count": 6,
|
||||
"metadata": {
|
||||
"collapsed": true
|
||||
},
|
||||
"outputs": [],
|
||||
"source": [
|
||||
"# tf Graph input\n",
|
||||
"x = tf.placeholder(tf.types.float32, [None, n_input])\n",
|
||||
"y = tf.placeholder(tf.types.float32, [None, n_classes])\n",
|
||||
"keep_prob = tf.placeholder(tf.types.float32) #dropout (keep probability)"
|
||||
]
|
||||
},
|
||||
{
|
||||
"cell_type": "code",
|
||||
"execution_count": 8,
|
||||
"metadata": {
|
||||
"collapsed": true
|
||||
},
|
||||
"outputs": [],
|
||||
"source": [
|
||||
"# Create model\n",
|
||||
"def conv2d(img, w, b):\n",
|
||||
" return tf.nn.relu(tf.nn.bias_add(tf.nn.conv2d(img, w, strides=[1, 1, 1, 1], \n",
|
||||
" padding='SAME'),b))\n",
|
||||
"\n",
|
||||
"def max_pool(img, k):\n",
|
||||
" return tf.nn.max_pool(img, ksize=[1, k, k, 1], strides=[1, k, k, 1], padding='SAME')\n",
|
||||
"\n",
|
||||
"def conv_net(_X, _weights, _biases, _dropout):\n",
|
||||
" # Reshape input picture\n",
|
||||
" _X = tf.reshape(_X, shape=[-1, 28, 28, 1])\n",
|
||||
"\n",
|
||||
" # Convolution Layer\n",
|
||||
" conv1 = conv2d(_X, _weights['wc1'], _biases['bc1'])\n",
|
||||
" # Max Pooling (down-sampling)\n",
|
||||
" conv1 = max_pool(conv1, k=2)\n",
|
||||
" # Apply Dropout\n",
|
||||
" conv1 = tf.nn.dropout(conv1, _dropout)\n",
|
||||
"\n",
|
||||
" # Convolution Layer\n",
|
||||
" conv2 = conv2d(conv1, _weights['wc2'], _biases['bc2'])\n",
|
||||
" # Max Pooling (down-sampling)\n",
|
||||
" conv2 = max_pool(conv2, k=2)\n",
|
||||
" # Apply Dropout\n",
|
||||
" conv2 = tf.nn.dropout(conv2, _dropout)\n",
|
||||
"\n",
|
||||
" # Fully connected layer\n",
|
||||
" # Reshape conv2 output to fit dense layer input\n",
|
||||
" dense1 = tf.reshape(conv2, [-1, _weights['wd1'].get_shape().as_list()[0]]) \n",
|
||||
" # Relu activation\n",
|
||||
" dense1 = tf.nn.relu(tf.add(tf.matmul(dense1, _weights['wd1']), _biases['bd1']))\n",
|
||||
" # Apply Dropout\n",
|
||||
" dense1 = tf.nn.dropout(dense1, _dropout) # Apply Dropout\n",
|
||||
"\n",
|
||||
" # Output, class prediction\n",
|
||||
" out = tf.add(tf.matmul(dense1, _weights['out']), _biases['out'])\n",
|
||||
" return out"
|
||||
]
|
||||
},
|
||||
{
|
||||
"cell_type": "code",
|
||||
"execution_count": 9,
|
||||
"metadata": {
|
||||
"collapsed": true
|
||||
},
|
||||
"outputs": [],
|
||||
"source": [
|
||||
"# Store layers weight & bias\n",
|
||||
"weights = {\n",
|
||||
" # 5x5 conv, 1 input, 32 outputs\n",
|
||||
" 'wc1': tf.Variable(tf.random_normal([5, 5, 1, 32])), \n",
|
||||
" # 5x5 conv, 32 inputs, 64 outputs\n",
|
||||
" 'wc2': tf.Variable(tf.random_normal([5, 5, 32, 64])), \n",
|
||||
" # fully connected, 7*7*64 inputs, 1024 outputs\n",
|
||||
" 'wd1': tf.Variable(tf.random_normal([7*7*64, 1024])), \n",
|
||||
" # 1024 inputs, 10 outputs (class prediction)\n",
|
||||
" 'out': tf.Variable(tf.random_normal([1024, n_classes])) \n",
|
||||
"}\n",
|
||||
"\n",
|
||||
"biases = {\n",
|
||||
" 'bc1': tf.Variable(tf.random_normal([32])),\n",
|
||||
" 'bc2': tf.Variable(tf.random_normal([64])),\n",
|
||||
" 'bd1': tf.Variable(tf.random_normal([1024])),\n",
|
||||
" 'out': tf.Variable(tf.random_normal([n_classes]))\n",
|
||||
"}"
|
||||
]
|
||||
},
|
||||
{
|
||||
"cell_type": "code",
|
||||
"execution_count": 10,
|
||||
"metadata": {
|
||||
"collapsed": true
|
||||
},
|
||||
"outputs": [],
|
||||
"source": [
|
||||
"# Construct model\n",
|
||||
"pred = conv_net(x, weights, biases, keep_prob)"
|
||||
]
|
||||
},
|
||||
{
|
||||
"cell_type": "code",
|
||||
"execution_count": 11,
|
||||
"metadata": {
|
||||
"collapsed": true
|
||||
},
|
||||
"outputs": [],
|
||||
"source": [
|
||||
"# Define loss and optimizer\n",
|
||||
"cost = tf.reduce_mean(tf.nn.softmax_cross_entropy_with_logits(pred, y))\n",
|
||||
"optimizer = tf.train.AdamOptimizer(learning_rate=learning_rate).minimize(cost)"
|
||||
]
|
||||
},
|
||||
{
|
||||
"cell_type": "code",
|
||||
"execution_count": 12,
|
||||
"metadata": {
|
||||
"collapsed": true
|
||||
},
|
||||
"outputs": [],
|
||||
"source": [
|
||||
"# Evaluate model\n",
|
||||
"correct_pred = tf.equal(tf.argmax(pred,1), tf.argmax(y,1))\n",
|
||||
"accuracy = tf.reduce_mean(tf.cast(correct_pred, tf.types.float32))"
|
||||
]
|
||||
},
|
||||
{
|
||||
"cell_type": "code",
|
||||
"execution_count": 13,
|
||||
"metadata": {
|
||||
"collapsed": true
|
||||
},
|
||||
"outputs": [],
|
||||
"source": [
|
||||
"# Initializing the variables\n",
|
||||
"init = tf.initialize_all_variables()"
|
||||
]
|
||||
},
|
||||
{
|
||||
"cell_type": "code",
|
||||
"execution_count": 19,
|
||||
"metadata": {
|
||||
"collapsed": false
|
||||
},
|
||||
"outputs": [
|
||||
{
|
||||
"name": "stdout",
|
||||
"output_type": "stream",
|
||||
"text": [
|
||||
"Iter 2560, Minibatch Loss= 26046.011719, Training Accuracy= 0.21094\n",
|
||||
"Iter 5120, Minibatch Loss= 10456.769531, Training Accuracy= 0.52344\n",
|
||||
"Iter 7680, Minibatch Loss= 6273.207520, Training Accuracy= 0.71875\n",
|
||||
"Iter 10240, Minibatch Loss= 6276.231445, Training Accuracy= 0.64062\n",
|
||||
"Iter 12800, Minibatch Loss= 4188.221680, Training Accuracy= 0.77344\n",
|
||||
"Iter 15360, Minibatch Loss= 2717.077637, Training Accuracy= 0.80469\n",
|
||||
"Iter 17920, Minibatch Loss= 4057.120361, Training Accuracy= 0.81250\n",
|
||||
"Iter 20480, Minibatch Loss= 1696.550415, Training Accuracy= 0.87500\n",
|
||||
"Iter 23040, Minibatch Loss= 2525.317627, Training Accuracy= 0.85938\n",
|
||||
"Iter 25600, Minibatch Loss= 2341.906738, Training Accuracy= 0.87500\n",
|
||||
"Iter 28160, Minibatch Loss= 4200.535156, Training Accuracy= 0.79688\n",
|
||||
"Iter 30720, Minibatch Loss= 1888.964355, Training Accuracy= 0.89062\n",
|
||||
"Iter 33280, Minibatch Loss= 2167.645996, Training Accuracy= 0.84375\n",
|
||||
"Iter 35840, Minibatch Loss= 1932.107544, Training Accuracy= 0.89844\n",
|
||||
"Iter 38400, Minibatch Loss= 1562.430054, Training Accuracy= 0.90625\n",
|
||||
"Iter 40960, Minibatch Loss= 1676.755249, Training Accuracy= 0.84375\n",
|
||||
"Iter 43520, Minibatch Loss= 1003.626099, Training Accuracy= 0.93750\n",
|
||||
"Iter 46080, Minibatch Loss= 1176.615479, Training Accuracy= 0.86719\n",
|
||||
"Iter 48640, Minibatch Loss= 1260.592651, Training Accuracy= 0.88281\n",
|
||||
"Iter 51200, Minibatch Loss= 1399.667969, Training Accuracy= 0.86719\n",
|
||||
"Iter 53760, Minibatch Loss= 1259.961426, Training Accuracy= 0.89844\n",
|
||||
"Iter 56320, Minibatch Loss= 1415.800781, Training Accuracy= 0.89062\n",
|
||||
"Iter 58880, Minibatch Loss= 1835.365967, Training Accuracy= 0.85156\n",
|
||||
"Iter 61440, Minibatch Loss= 1395.168823, Training Accuracy= 0.90625\n",
|
||||
"Iter 64000, Minibatch Loss= 973.283569, Training Accuracy= 0.88281\n",
|
||||
"Iter 66560, Minibatch Loss= 818.093811, Training Accuracy= 0.92969\n",
|
||||
"Iter 69120, Minibatch Loss= 1178.744263, Training Accuracy= 0.92188\n",
|
||||
"Iter 71680, Minibatch Loss= 845.889709, Training Accuracy= 0.89844\n",
|
||||
"Iter 74240, Minibatch Loss= 1259.505615, Training Accuracy= 0.90625\n",
|
||||
"Iter 76800, Minibatch Loss= 738.037109, Training Accuracy= 0.89844\n",
|
||||
"Iter 79360, Minibatch Loss= 862.499146, Training Accuracy= 0.93750\n",
|
||||
"Iter 81920, Minibatch Loss= 739.704041, Training Accuracy= 0.90625\n",
|
||||
"Iter 84480, Minibatch Loss= 652.880310, Training Accuracy= 0.95312\n",
|
||||
"Iter 87040, Minibatch Loss= 635.464600, Training Accuracy= 0.92969\n",
|
||||
"Iter 89600, Minibatch Loss= 933.166626, Training Accuracy= 0.90625\n",
|
||||
"Iter 92160, Minibatch Loss= 213.874893, Training Accuracy= 0.96094\n",
|
||||
"Iter 94720, Minibatch Loss= 609.575684, Training Accuracy= 0.91406\n",
|
||||
"Iter 97280, Minibatch Loss= 560.208008, Training Accuracy= 0.93750\n",
|
||||
"Iter 99840, Minibatch Loss= 963.577148, Training Accuracy= 0.90625\n",
|
||||
"Optimization Finished!\n",
|
||||
"Testing Accuracy: 0.960938\n"
|
||||
]
|
||||
}
|
||||
],
|
||||
"source": [
|
||||
"# Launch the graph\n",
|
||||
"with tf.Session() as sess:\n",
|
||||
" sess.run(init)\n",
|
||||
" step = 1\n",
|
||||
" # Keep training until reach max iterations\n",
|
||||
" while step * batch_size < training_iters:\n",
|
||||
" batch_xs, batch_ys = mnist.train.next_batch(batch_size)\n",
|
||||
" # Fit training using batch data\n",
|
||||
" sess.run(optimizer, feed_dict={x: batch_xs, y: batch_ys, keep_prob: dropout})\n",
|
||||
" if step % display_step == 0:\n",
|
||||
" # Calculate batch accuracy\n",
|
||||
" acc = sess.run(accuracy, feed_dict={x: batch_xs, y: batch_ys, keep_prob: 1.})\n",
|
||||
" # Calculate batch loss\n",
|
||||
" loss = sess.run(cost, feed_dict={x: batch_xs, y: batch_ys, keep_prob: 1.})\n",
|
||||
" print \"Iter \" + str(step*batch_size) + \", Minibatch Loss= \" + \\\n",
|
||||
" \"{:.6f}\".format(loss) + \", Training Accuracy= \" + \"{:.5f}\".format(acc)\n",
|
||||
" step += 1\n",
|
||||
" print \"Optimization Finished!\"\n",
|
||||
" # Calculate accuracy for 256 mnist test images\n",
|
||||
" print \"Testing Accuracy:\", sess.run(accuracy, feed_dict={x: mnist.test.images[:256], \n",
|
||||
" y: mnist.test.labels[:256], \n",
|
||||
" keep_prob: 1.})"
|
||||
]
|
||||
}
|
||||
],
|
||||
"metadata": {
|
||||
"kernelspec": {
|
||||
"display_name": "Python 3",
|
||||
"language": "python",
|
||||
"name": "python3"
|
||||
},
|
||||
"language_info": {
|
||||
"codemirror_mode": {
|
||||
"name": "ipython",
|
||||
"version": 3
|
||||
},
|
||||
"file_extension": ".py",
|
||||
"mimetype": "text/x-python",
|
||||
"name": "python",
|
||||
"nbconvert_exporter": "python",
|
||||
"pygments_lexer": "ipython3",
|
||||
"version": "3.4.3"
|
||||
}
|
||||
},
|
||||
"nbformat": 4,
|
||||
"nbformat_minor": 0
|
||||
}
|
||||
|
|
@ -0,0 +1,253 @@
|
|||
{
|
||||
"cells": [
|
||||
{
|
||||
"cell_type": "markdown",
|
||||
"metadata": {},
|
||||
"source": [
|
||||
"# Multilayer Perceptron in TensorFlow\n",
|
||||
"\n",
|
||||
"Credits: Forked from [TensorFlow-Examples](https://github.com/aymericdamien/TensorFlow-Examples) by Aymeric Damien\n",
|
||||
"\n",
|
||||
"## Setup\n",
|
||||
"\n",
|
||||
"Refer to the [setup instructions](http://nbviewer.ipython.org/github/donnemartin/data-science-ipython-notebooks/blob/master/deep-learning/tensor-flow-examples/Setup_TensorFlow.md)"
|
||||
]
|
||||
},
|
||||
{
|
||||
"cell_type": "code",
|
||||
"execution_count": 2,
|
||||
"metadata": {
|
||||
"collapsed": false
|
||||
},
|
||||
"outputs": [
|
||||
{
|
||||
"name": "stdout",
|
||||
"output_type": "stream",
|
||||
"text": [
|
||||
"Extracting /tmp/data/train-images-idx3-ubyte.gz\n",
|
||||
"Extracting /tmp/data/train-labels-idx1-ubyte.gz\n",
|
||||
"Extracting /tmp/data/t10k-images-idx3-ubyte.gz\n",
|
||||
"Extracting /tmp/data/t10k-labels-idx1-ubyte.gz\n"
|
||||
]
|
||||
}
|
||||
],
|
||||
"source": [
|
||||
"# Import MINST data\n",
|
||||
"import input_data\n",
|
||||
"mnist = input_data.read_data_sets(\"/tmp/data/\", one_hot=True)"
|
||||
]
|
||||
},
|
||||
{
|
||||
"cell_type": "code",
|
||||
"execution_count": 3,
|
||||
"metadata": {
|
||||
"collapsed": true
|
||||
},
|
||||
"outputs": [],
|
||||
"source": [
|
||||
"import tensorflow as tf"
|
||||
]
|
||||
},
|
||||
{
|
||||
"cell_type": "code",
|
||||
"execution_count": 4,
|
||||
"metadata": {
|
||||
"collapsed": true
|
||||
},
|
||||
"outputs": [],
|
||||
"source": [
|
||||
"# Parameters\n",
|
||||
"learning_rate = 0.001\n",
|
||||
"training_epochs = 15\n",
|
||||
"batch_size = 100\n",
|
||||
"display_step = 1"
|
||||
]
|
||||
},
|
||||
{
|
||||
"cell_type": "code",
|
||||
"execution_count": 5,
|
||||
"metadata": {
|
||||
"collapsed": true
|
||||
},
|
||||
"outputs": [],
|
||||
"source": [
|
||||
"# Network Parameters\n",
|
||||
"n_hidden_1 = 256 # 1st layer num features\n",
|
||||
"n_hidden_2 = 256 # 2nd layer num features\n",
|
||||
"n_input = 784 # MNIST data input (img shape: 28*28)\n",
|
||||
"n_classes = 10 # MNIST total classes (0-9 digits)"
|
||||
]
|
||||
},
|
||||
{
|
||||
"cell_type": "code",
|
||||
"execution_count": 6,
|
||||
"metadata": {
|
||||
"collapsed": true
|
||||
},
|
||||
"outputs": [],
|
||||
"source": [
|
||||
"# tf Graph input\n",
|
||||
"x = tf.placeholder(\"float\", [None, n_input])\n",
|
||||
"y = tf.placeholder(\"float\", [None, n_classes])"
|
||||
]
|
||||
},
|
||||
{
|
||||
"cell_type": "code",
|
||||
"execution_count": 7,
|
||||
"metadata": {
|
||||
"collapsed": true
|
||||
},
|
||||
"outputs": [],
|
||||
"source": [
|
||||
"# Create model\n",
|
||||
"def multilayer_perceptron(_X, _weights, _biases):\n",
|
||||
" #Hidden layer with RELU activation\n",
|
||||
" layer_1 = tf.nn.relu(tf.add(tf.matmul(_X, _weights['h1']), _biases['b1'])) \n",
|
||||
" #Hidden layer with RELU activation\n",
|
||||
" layer_2 = tf.nn.relu(tf.add(tf.matmul(layer_1, _weights['h2']), _biases['b2'])) \n",
|
||||
" return tf.matmul(layer_2, weights['out']) + biases['out']"
|
||||
]
|
||||
},
|
||||
{
|
||||
"cell_type": "code",
|
||||
"execution_count": 8,
|
||||
"metadata": {
|
||||
"collapsed": true
|
||||
},
|
||||
"outputs": [],
|
||||
"source": [
|
||||
"# Store layers weight & bias\n",
|
||||
"weights = {\n",
|
||||
" 'h1': tf.Variable(tf.random_normal([n_input, n_hidden_1])),\n",
|
||||
" 'h2': tf.Variable(tf.random_normal([n_hidden_1, n_hidden_2])),\n",
|
||||
" 'out': tf.Variable(tf.random_normal([n_hidden_2, n_classes]))\n",
|
||||
"}\n",
|
||||
"biases = {\n",
|
||||
" 'b1': tf.Variable(tf.random_normal([n_hidden_1])),\n",
|
||||
" 'b2': tf.Variable(tf.random_normal([n_hidden_2])),\n",
|
||||
" 'out': tf.Variable(tf.random_normal([n_classes]))\n",
|
||||
"}"
|
||||
]
|
||||
},
|
||||
{
|
||||
"cell_type": "code",
|
||||
"execution_count": 9,
|
||||
"metadata": {
|
||||
"collapsed": true
|
||||
},
|
||||
"outputs": [],
|
||||
"source": [
|
||||
"# Construct model\n",
|
||||
"pred = multilayer_perceptron(x, weights, biases)"
|
||||
]
|
||||
},
|
||||
{
|
||||
"cell_type": "code",
|
||||
"execution_count": 10,
|
||||
"metadata": {
|
||||
"collapsed": true
|
||||
},
|
||||
"outputs": [],
|
||||
"source": [
|
||||
"# Define loss and optimizer\n",
|
||||
"# Softmax loss\n",
|
||||
"cost = tf.reduce_mean(tf.nn.softmax_cross_entropy_with_logits(pred, y)) \n",
|
||||
"# Adam Optimizer\n",
|
||||
"optimizer = tf.train.AdamOptimizer(learning_rate=learning_rate).minimize(cost) "
|
||||
]
|
||||
},
|
||||
{
|
||||
"cell_type": "code",
|
||||
"execution_count": 11,
|
||||
"metadata": {
|
||||
"collapsed": true
|
||||
},
|
||||
"outputs": [],
|
||||
"source": [
|
||||
"# Initializing the variables\n",
|
||||
"init = tf.initialize_all_variables()"
|
||||
]
|
||||
},
|
||||
{
|
||||
"cell_type": "code",
|
||||
"execution_count": 12,
|
||||
"metadata": {
|
||||
"collapsed": false
|
||||
},
|
||||
"outputs": [
|
||||
{
|
||||
"name": "stdout",
|
||||
"output_type": "stream",
|
||||
"text": [
|
||||
"Epoch: 0001 cost= 160.113980416\n",
|
||||
"Epoch: 0002 cost= 38.665780694\n",
|
||||
"Epoch: 0003 cost= 24.118004577\n",
|
||||
"Epoch: 0004 cost= 16.440921303\n",
|
||||
"Epoch: 0005 cost= 11.689460141\n",
|
||||
"Epoch: 0006 cost= 8.469423468\n",
|
||||
"Epoch: 0007 cost= 6.223237230\n",
|
||||
"Epoch: 0008 cost= 4.560174118\n",
|
||||
"Epoch: 0009 cost= 3.250516910\n",
|
||||
"Epoch: 0010 cost= 2.359658795\n",
|
||||
"Epoch: 0011 cost= 1.694081847\n",
|
||||
"Epoch: 0012 cost= 1.167997509\n",
|
||||
"Epoch: 0013 cost= 0.872986831\n",
|
||||
"Epoch: 0014 cost= 0.630616366\n",
|
||||
"Epoch: 0015 cost= 0.487381571\n",
|
||||
"Optimization Finished!\n",
|
||||
"Accuracy: 0.9462\n"
|
||||
]
|
||||
}
|
||||
],
|
||||
"source": [
|
||||
"# Launch the graph\n",
|
||||
"with tf.Session() as sess:\n",
|
||||
" sess.run(init)\n",
|
||||
"\n",
|
||||
" # Training cycle\n",
|
||||
" for epoch in range(training_epochs):\n",
|
||||
" avg_cost = 0.\n",
|
||||
" total_batch = int(mnist.train.num_examples/batch_size)\n",
|
||||
" # Loop over all batches\n",
|
||||
" for i in range(total_batch):\n",
|
||||
" batch_xs, batch_ys = mnist.train.next_batch(batch_size)\n",
|
||||
" # Fit training using batch data\n",
|
||||
" sess.run(optimizer, feed_dict={x: batch_xs, y: batch_ys})\n",
|
||||
" # Compute average loss\n",
|
||||
" avg_cost += sess.run(cost, feed_dict={x: batch_xs, y: batch_ys})/total_batch\n",
|
||||
" # Display logs per epoch step\n",
|
||||
" if epoch % display_step == 0:\n",
|
||||
" print \"Epoch:\", '%04d' % (epoch+1), \"cost=\", \"{:.9f}\".format(avg_cost)\n",
|
||||
"\n",
|
||||
" print \"Optimization Finished!\"\n",
|
||||
"\n",
|
||||
" # Test model\n",
|
||||
" correct_prediction = tf.equal(tf.argmax(pred, 1), tf.argmax(y, 1))\n",
|
||||
" # Calculate accuracy\n",
|
||||
" accuracy = tf.reduce_mean(tf.cast(correct_prediction, \"float\"))\n",
|
||||
" print \"Accuracy:\", accuracy.eval({x: mnist.test.images, y: mnist.test.labels})"
|
||||
]
|
||||
}
|
||||
],
|
||||
"metadata": {
|
||||
"kernelspec": {
|
||||
"display_name": "Python 3",
|
||||
"language": "python",
|
||||
"name": "python3"
|
||||
},
|
||||
"language_info": {
|
||||
"codemirror_mode": {
|
||||
"name": "ipython",
|
||||
"version": 3
|
||||
},
|
||||
"file_extension": ".py",
|
||||
"mimetype": "text/x-python",
|
||||
"name": "python",
|
||||
"nbconvert_exporter": "python",
|
||||
"pygments_lexer": "ipython3",
|
||||
"version": "3.4.3"
|
||||
}
|
||||
},
|
||||
"nbformat": 4,
|
||||
"nbformat_minor": 0
|
||||
}
|
||||
|
|
@ -0,0 +1,294 @@
|
|||
{
|
||||
"cells": [
|
||||
{
|
||||
"cell_type": "markdown",
|
||||
"metadata": {},
|
||||
"source": [
|
||||
"# Recurrent Neural Network in TensorFlow\n",
|
||||
"\n",
|
||||
"Credits: Forked from [TensorFlow-Examples](https://github.com/aymericdamien/TensorFlow-Examples) by Aymeric Damien\n",
|
||||
"\n",
|
||||
"## Setup\n",
|
||||
"\n",
|
||||
"Refer to the [setup instructions](http://nbviewer.ipython.org/github/donnemartin/data-science-ipython-notebooks/blob/master/deep-learning/tensor-flow-examples/Setup_TensorFlow.md)"
|
||||
]
|
||||
},
|
||||
{
|
||||
"cell_type": "code",
|
||||
"execution_count": 2,
|
||||
"metadata": {
|
||||
"collapsed": false
|
||||
},
|
||||
"outputs": [
|
||||
{
|
||||
"name": "stdout",
|
||||
"output_type": "stream",
|
||||
"text": [
|
||||
"Extracting /tmp/data/train-images-idx3-ubyte.gz\n",
|
||||
"Extracting /tmp/data/train-labels-idx1-ubyte.gz\n",
|
||||
"Extracting /tmp/data/t10k-images-idx3-ubyte.gz\n",
|
||||
"Extracting /tmp/data/t10k-labels-idx1-ubyte.gz\n"
|
||||
]
|
||||
}
|
||||
],
|
||||
"source": [
|
||||
"# Import MINST data\n",
|
||||
"import input_data\n",
|
||||
"mnist = input_data.read_data_sets(\"/tmp/data/\", one_hot=True)\n",
|
||||
"\n",
|
||||
"import tensorflow as tf\n",
|
||||
"from tensorflow.models.rnn import rnn, rnn_cell\n",
|
||||
"import numpy as np"
|
||||
]
|
||||
},
|
||||
{
|
||||
"cell_type": "code",
|
||||
"execution_count": 3,
|
||||
"metadata": {
|
||||
"collapsed": true
|
||||
},
|
||||
"outputs": [],
|
||||
"source": [
|
||||
"'''\n",
|
||||
"To classify images using a reccurent neural network, we consider every image row as a sequence of pixels.\n",
|
||||
"Because MNIST image shape is 28*28px, we will then handle 28 sequences of 28 steps for every sample.\n",
|
||||
"'''\n",
|
||||
"\n",
|
||||
"# Parameters\n",
|
||||
"learning_rate = 0.001\n",
|
||||
"training_iters = 100000\n",
|
||||
"batch_size = 128\n",
|
||||
"display_step = 10\n",
|
||||
"\n",
|
||||
"# Network Parameters\n",
|
||||
"n_input = 28 # MNIST data input (img shape: 28*28)\n",
|
||||
"n_steps = 28 # timesteps\n",
|
||||
"n_hidden = 128 # hidden layer num of features\n",
|
||||
"n_classes = 10 # MNIST total classes (0-9 digits)"
|
||||
]
|
||||
},
|
||||
{
|
||||
"cell_type": "code",
|
||||
"execution_count": 4,
|
||||
"metadata": {
|
||||
"collapsed": true
|
||||
},
|
||||
"outputs": [],
|
||||
"source": [
|
||||
"# tf Graph input\n",
|
||||
"x = tf.placeholder(\"float\", [None, n_steps, n_input])\n",
|
||||
"istate = tf.placeholder(\"float\", [None, 2*n_hidden]) #state & cell => 2x n_hidden\n",
|
||||
"y = tf.placeholder(\"float\", [None, n_classes])\n",
|
||||
"\n",
|
||||
"# Define weights\n",
|
||||
"weights = {\n",
|
||||
" 'hidden': tf.Variable(tf.random_normal([n_input, n_hidden])), # Hidden layer weights\n",
|
||||
" 'out': tf.Variable(tf.random_normal([n_hidden, n_classes]))\n",
|
||||
"}\n",
|
||||
"biases = {\n",
|
||||
" 'hidden': tf.Variable(tf.random_normal([n_hidden])),\n",
|
||||
" 'out': tf.Variable(tf.random_normal([n_classes]))\n",
|
||||
"}"
|
||||
]
|
||||
},
|
||||
{
|
||||
"cell_type": "code",
|
||||
"execution_count": 5,
|
||||
"metadata": {
|
||||
"collapsed": true
|
||||
},
|
||||
"outputs": [],
|
||||
"source": [
|
||||
"def RNN(_X, _istate, _weights, _biases):\n",
|
||||
"\n",
|
||||
" # input shape: (batch_size, n_steps, n_input)\n",
|
||||
" _X = tf.transpose(_X, [1, 0, 2]) # permute n_steps and batch_size\n",
|
||||
" # Reshape to prepare input to hidden activation\n",
|
||||
" _X = tf.reshape(_X, [-1, n_input]) # (n_steps*batch_size, n_input)\n",
|
||||
" # Linear activation\n",
|
||||
" _X = tf.matmul(_X, _weights['hidden']) + _biases['hidden']\n",
|
||||
"\n",
|
||||
" # Define a lstm cell with tensorflow\n",
|
||||
" lstm_cell = rnn_cell.BasicLSTMCell(n_hidden, forget_bias=1.0)\n",
|
||||
" # Split data because rnn cell needs a list of inputs for the RNN inner loop\n",
|
||||
" _X = tf.split(0, n_steps, _X) # n_steps * (batch_size, n_hidden)\n",
|
||||
"\n",
|
||||
" # Get lstm cell output\n",
|
||||
" outputs, states = rnn.rnn(lstm_cell, _X, initial_state=_istate)\n",
|
||||
"\n",
|
||||
" # Linear activation\n",
|
||||
" # Get inner loop last output\n",
|
||||
" return tf.matmul(outputs[-1], _weights['out']) + _biases['out']"
|
||||
]
|
||||
},
|
||||
{
|
||||
"cell_type": "code",
|
||||
"execution_count": 6,
|
||||
"metadata": {
|
||||
"collapsed": false
|
||||
},
|
||||
"outputs": [],
|
||||
"source": [
|
||||
"pred = RNN(x, istate, weights, biases)\n",
|
||||
"\n",
|
||||
"# Define loss and optimizer\n",
|
||||
"cost = tf.reduce_mean(tf.nn.softmax_cross_entropy_with_logits(pred, y)) # Softmax loss\n",
|
||||
"optimizer = tf.train.AdamOptimizer(learning_rate=learning_rate).minimize(cost) # Adam Optimizer\n",
|
||||
"\n",
|
||||
"# Evaluate model\n",
|
||||
"correct_pred = tf.equal(tf.argmax(pred,1), tf.argmax(y,1))\n",
|
||||
"accuracy = tf.reduce_mean(tf.cast(correct_pred, tf.types.float32))"
|
||||
]
|
||||
},
|
||||
{
|
||||
"cell_type": "code",
|
||||
"execution_count": 7,
|
||||
"metadata": {
|
||||
"collapsed": false
|
||||
},
|
||||
"outputs": [
|
||||
{
|
||||
"name": "stdout",
|
||||
"output_type": "stream",
|
||||
"text": [
|
||||
"Iter 1280, Minibatch Loss= 1.888242, Training Accuracy= 0.39844\n",
|
||||
"Iter 2560, Minibatch Loss= 1.519879, Training Accuracy= 0.47656\n",
|
||||
"Iter 3840, Minibatch Loss= 1.238005, Training Accuracy= 0.63281\n",
|
||||
"Iter 5120, Minibatch Loss= 0.933760, Training Accuracy= 0.71875\n",
|
||||
"Iter 6400, Minibatch Loss= 0.832130, Training Accuracy= 0.73438\n",
|
||||
"Iter 7680, Minibatch Loss= 0.979760, Training Accuracy= 0.70312\n",
|
||||
"Iter 8960, Minibatch Loss= 0.821921, Training Accuracy= 0.71875\n",
|
||||
"Iter 10240, Minibatch Loss= 0.710566, Training Accuracy= 0.79688\n",
|
||||
"Iter 11520, Minibatch Loss= 0.578501, Training Accuracy= 0.82812\n",
|
||||
"Iter 12800, Minibatch Loss= 0.765049, Training Accuracy= 0.75000\n",
|
||||
"Iter 14080, Minibatch Loss= 0.582995, Training Accuracy= 0.78125\n",
|
||||
"Iter 15360, Minibatch Loss= 0.575092, Training Accuracy= 0.79688\n",
|
||||
"Iter 16640, Minibatch Loss= 0.701214, Training Accuracy= 0.75781\n",
|
||||
"Iter 17920, Minibatch Loss= 0.561972, Training Accuracy= 0.78125\n",
|
||||
"Iter 19200, Minibatch Loss= 0.394480, Training Accuracy= 0.85938\n",
|
||||
"Iter 20480, Minibatch Loss= 0.356244, Training Accuracy= 0.91406\n",
|
||||
"Iter 21760, Minibatch Loss= 0.632163, Training Accuracy= 0.78125\n",
|
||||
"Iter 23040, Minibatch Loss= 0.269334, Training Accuracy= 0.90625\n",
|
||||
"Iter 24320, Minibatch Loss= 0.485007, Training Accuracy= 0.86719\n",
|
||||
"Iter 25600, Minibatch Loss= 0.569704, Training Accuracy= 0.78906\n",
|
||||
"Iter 26880, Minibatch Loss= 0.267697, Training Accuracy= 0.92188\n",
|
||||
"Iter 28160, Minibatch Loss= 0.381177, Training Accuracy= 0.90625\n",
|
||||
"Iter 29440, Minibatch Loss= 0.350800, Training Accuracy= 0.87500\n",
|
||||
"Iter 30720, Minibatch Loss= 0.356782, Training Accuracy= 0.90625\n",
|
||||
"Iter 32000, Minibatch Loss= 0.322511, Training Accuracy= 0.89062\n",
|
||||
"Iter 33280, Minibatch Loss= 0.309195, Training Accuracy= 0.90625\n",
|
||||
"Iter 34560, Minibatch Loss= 0.535408, Training Accuracy= 0.83594\n",
|
||||
"Iter 35840, Minibatch Loss= 0.281643, Training Accuracy= 0.92969\n",
|
||||
"Iter 37120, Minibatch Loss= 0.290962, Training Accuracy= 0.89844\n",
|
||||
"Iter 38400, Minibatch Loss= 0.204718, Training Accuracy= 0.93750\n",
|
||||
"Iter 39680, Minibatch Loss= 0.205882, Training Accuracy= 0.92969\n",
|
||||
"Iter 40960, Minibatch Loss= 0.481441, Training Accuracy= 0.84375\n",
|
||||
"Iter 42240, Minibatch Loss= 0.348245, Training Accuracy= 0.89844\n",
|
||||
"Iter 43520, Minibatch Loss= 0.274692, Training Accuracy= 0.90625\n",
|
||||
"Iter 44800, Minibatch Loss= 0.171815, Training Accuracy= 0.94531\n",
|
||||
"Iter 46080, Minibatch Loss= 0.171035, Training Accuracy= 0.93750\n",
|
||||
"Iter 47360, Minibatch Loss= 0.235800, Training Accuracy= 0.89844\n",
|
||||
"Iter 48640, Minibatch Loss= 0.235974, Training Accuracy= 0.93750\n",
|
||||
"Iter 49920, Minibatch Loss= 0.207323, Training Accuracy= 0.92188\n",
|
||||
"Iter 51200, Minibatch Loss= 0.212989, Training Accuracy= 0.91406\n",
|
||||
"Iter 52480, Minibatch Loss= 0.151774, Training Accuracy= 0.95312\n",
|
||||
"Iter 53760, Minibatch Loss= 0.090070, Training Accuracy= 0.96875\n",
|
||||
"Iter 55040, Minibatch Loss= 0.264714, Training Accuracy= 0.92969\n",
|
||||
"Iter 56320, Minibatch Loss= 0.235086, Training Accuracy= 0.92969\n",
|
||||
"Iter 57600, Minibatch Loss= 0.160302, Training Accuracy= 0.95312\n",
|
||||
"Iter 58880, Minibatch Loss= 0.106515, Training Accuracy= 0.96875\n",
|
||||
"Iter 60160, Minibatch Loss= 0.236039, Training Accuracy= 0.94531\n",
|
||||
"Iter 61440, Minibatch Loss= 0.279540, Training Accuracy= 0.90625\n",
|
||||
"Iter 62720, Minibatch Loss= 0.173585, Training Accuracy= 0.93750\n",
|
||||
"Iter 64000, Minibatch Loss= 0.191009, Training Accuracy= 0.92188\n",
|
||||
"Iter 65280, Minibatch Loss= 0.210331, Training Accuracy= 0.89844\n",
|
||||
"Iter 66560, Minibatch Loss= 0.223444, Training Accuracy= 0.94531\n",
|
||||
"Iter 67840, Minibatch Loss= 0.278210, Training Accuracy= 0.91406\n",
|
||||
"Iter 69120, Minibatch Loss= 0.174290, Training Accuracy= 0.95312\n",
|
||||
"Iter 70400, Minibatch Loss= 0.188701, Training Accuracy= 0.94531\n",
|
||||
"Iter 71680, Minibatch Loss= 0.210277, Training Accuracy= 0.94531\n",
|
||||
"Iter 72960, Minibatch Loss= 0.249951, Training Accuracy= 0.95312\n",
|
||||
"Iter 74240, Minibatch Loss= 0.209853, Training Accuracy= 0.92188\n",
|
||||
"Iter 75520, Minibatch Loss= 0.049742, Training Accuracy= 0.99219\n",
|
||||
"Iter 76800, Minibatch Loss= 0.250095, Training Accuracy= 0.92969\n",
|
||||
"Iter 78080, Minibatch Loss= 0.133853, Training Accuracy= 0.95312\n",
|
||||
"Iter 79360, Minibatch Loss= 0.110206, Training Accuracy= 0.97656\n",
|
||||
"Iter 80640, Minibatch Loss= 0.141906, Training Accuracy= 0.93750\n",
|
||||
"Iter 81920, Minibatch Loss= 0.126872, Training Accuracy= 0.94531\n",
|
||||
"Iter 83200, Minibatch Loss= 0.138925, Training Accuracy= 0.95312\n",
|
||||
"Iter 84480, Minibatch Loss= 0.128652, Training Accuracy= 0.96094\n",
|
||||
"Iter 85760, Minibatch Loss= 0.099837, Training Accuracy= 0.96094\n",
|
||||
"Iter 87040, Minibatch Loss= 0.119000, Training Accuracy= 0.95312\n",
|
||||
"Iter 88320, Minibatch Loss= 0.179807, Training Accuracy= 0.95312\n",
|
||||
"Iter 89600, Minibatch Loss= 0.141792, Training Accuracy= 0.96094\n",
|
||||
"Iter 90880, Minibatch Loss= 0.142424, Training Accuracy= 0.96094\n",
|
||||
"Iter 92160, Minibatch Loss= 0.159564, Training Accuracy= 0.96094\n",
|
||||
"Iter 93440, Minibatch Loss= 0.111984, Training Accuracy= 0.95312\n",
|
||||
"Iter 94720, Minibatch Loss= 0.238978, Training Accuracy= 0.92969\n",
|
||||
"Iter 96000, Minibatch Loss= 0.068002, Training Accuracy= 0.97656\n",
|
||||
"Iter 97280, Minibatch Loss= 0.191819, Training Accuracy= 0.94531\n",
|
||||
"Iter 98560, Minibatch Loss= 0.081197, Training Accuracy= 0.99219\n",
|
||||
"Iter 99840, Minibatch Loss= 0.206797, Training Accuracy= 0.95312\n",
|
||||
"Optimization Finished!\n",
|
||||
"Testing Accuracy: 0.941406\n"
|
||||
]
|
||||
}
|
||||
],
|
||||
"source": [
|
||||
"# Initializing the variables\n",
|
||||
"init = tf.initialize_all_variables()\n",
|
||||
"\n",
|
||||
"# Launch the graph\n",
|
||||
"with tf.Session() as sess:\n",
|
||||
" sess.run(init)\n",
|
||||
" step = 1\n",
|
||||
" # Keep training until reach max iterations\n",
|
||||
" while step * batch_size < training_iters:\n",
|
||||
" batch_xs, batch_ys = mnist.train.next_batch(batch_size)\n",
|
||||
" # Reshape data to get 28 seq of 28 elements\n",
|
||||
" batch_xs = batch_xs.reshape((batch_size, n_steps, n_input))\n",
|
||||
" # Fit training using batch data\n",
|
||||
" sess.run(optimizer, feed_dict={x: batch_xs, y: batch_ys,\n",
|
||||
" istate: np.zeros((batch_size, 2*n_hidden))})\n",
|
||||
" if step % display_step == 0:\n",
|
||||
" # Calculate batch accuracy\n",
|
||||
" acc = sess.run(accuracy, feed_dict={x: batch_xs, y: batch_ys,\n",
|
||||
" istate: np.zeros((batch_size, 2*n_hidden))})\n",
|
||||
" # Calculate batch loss\n",
|
||||
" loss = sess.run(cost, feed_dict={x: batch_xs, y: batch_ys,\n",
|
||||
" istate: np.zeros((batch_size, 2*n_hidden))})\n",
|
||||
" print \"Iter \" + str(step*batch_size) + \", Minibatch Loss= \" + \"{:.6f}\".format(loss) + \\\n",
|
||||
" \", Training Accuracy= \" + \"{:.5f}\".format(acc)\n",
|
||||
" step += 1\n",
|
||||
" print \"Optimization Finished!\"\n",
|
||||
" # Calculate accuracy for 256 mnist test images\n",
|
||||
" test_len = 256\n",
|
||||
" test_data = mnist.test.images[:test_len].reshape((-1, n_steps, n_input))\n",
|
||||
" test_label = mnist.test.labels[:test_len]\n",
|
||||
" print \"Testing Accuracy:\", sess.run(accuracy, feed_dict={x: test_data, y: test_label,\n",
|
||||
" istate: np.zeros((test_len, 2*n_hidden))})"
|
||||
]
|
||||
}
|
||||
],
|
||||
"metadata": {
|
||||
"kernelspec": {
|
||||
"display_name": "Python 3",
|
||||
"language": "python",
|
||||
"name": "python3"
|
||||
},
|
||||
"language_info": {
|
||||
"codemirror_mode": {
|
||||
"name": "ipython",
|
||||
"version": 3
|
||||
},
|
||||
"file_extension": ".py",
|
||||
"mimetype": "text/x-python",
|
||||
"name": "python",
|
||||
"nbconvert_exporter": "python",
|
||||
"pygments_lexer": "ipython3",
|
||||
"version": "3.4.3"
|
||||
}
|
||||
},
|
||||
"nbformat": 4,
|
||||
"nbformat_minor": 0
|
||||
}
|
||||
|
|
@ -0,0 +1,184 @@
|
|||
{
|
||||
"cells": [
|
||||
{
|
||||
"cell_type": "markdown",
|
||||
"metadata": {},
|
||||
"source": [
|
||||
"# Basic Multi GPU Computation in TensorFlow\n",
|
||||
"\n",
|
||||
"Credits: Forked from [TensorFlow-Examples](https://github.com/aymericdamien/TensorFlow-Examples) by Aymeric Damien\n",
|
||||
"\n",
|
||||
"## Setup\n",
|
||||
"\n",
|
||||
"Refer to the [setup instructions](http://nbviewer.ipython.org/github/donnemartin/data-science-ipython-notebooks/blob/master/deep-learning/tensor-flow-examples/Setup_TensorFlow.md)"
|
||||
]
|
||||
},
|
||||
{
|
||||
"cell_type": "markdown",
|
||||
"metadata": {
|
||||
"collapsed": true
|
||||
},
|
||||
"source": [
|
||||
"This tutorial requires your machine to have 2 GPUs\n",
|
||||
"* \"/cpu:0\": The CPU of your machine.\n",
|
||||
"* \"/gpu:0\": The first GPU of your machine\n",
|
||||
"* \"/gpu:1\": The second GPU of your machine\n",
|
||||
"* For this example, we are using 2 GTX-980"
|
||||
]
|
||||
},
|
||||
{
|
||||
"cell_type": "code",
|
||||
"execution_count": 2,
|
||||
"metadata": {
|
||||
"collapsed": true
|
||||
},
|
||||
"outputs": [],
|
||||
"source": [
|
||||
"import numpy as np\n",
|
||||
"import tensorflow as tf\n",
|
||||
"import datetime"
|
||||
]
|
||||
},
|
||||
{
|
||||
"cell_type": "code",
|
||||
"execution_count": 3,
|
||||
"metadata": {
|
||||
"collapsed": true
|
||||
},
|
||||
"outputs": [],
|
||||
"source": [
|
||||
"#Processing Units logs\n",
|
||||
"log_device_placement = True\n",
|
||||
"\n",
|
||||
"#num of multiplications to perform\n",
|
||||
"n = 10"
|
||||
]
|
||||
},
|
||||
{
|
||||
"cell_type": "code",
|
||||
"execution_count": null,
|
||||
"metadata": {
|
||||
"collapsed": false
|
||||
},
|
||||
"outputs": [],
|
||||
"source": [
|
||||
"# Example: compute A^n + B^n on 2 GPUs\n",
|
||||
"\n",
|
||||
"# Create random large matrix\n",
|
||||
"A = np.random.rand(1e4, 1e4).astype('float32')\n",
|
||||
"B = np.random.rand(1e4, 1e4).astype('float32')\n",
|
||||
"\n",
|
||||
"# Creates a graph to store results\n",
|
||||
"c1 = []\n",
|
||||
"c2 = []\n",
|
||||
"\n",
|
||||
"# Define matrix power\n",
|
||||
"def matpow(M, n):\n",
|
||||
" if n < 1: #Abstract cases where n < 1\n",
|
||||
" return M\n",
|
||||
" else:\n",
|
||||
" return tf.matmul(M, matpow(M, n-1))"
|
||||
]
|
||||
},
|
||||
{
|
||||
"cell_type": "code",
|
||||
"execution_count": 6,
|
||||
"metadata": {
|
||||
"collapsed": true
|
||||
},
|
||||
"outputs": [],
|
||||
"source": [
|
||||
"# Single GPU computing\n",
|
||||
"\n",
|
||||
"with tf.device('/gpu:0'):\n",
|
||||
" a = tf.constant(A)\n",
|
||||
" b = tf.constant(B)\n",
|
||||
" #compute A^n and B^n and store results in c1\n",
|
||||
" c1.append(matpow(a, n))\n",
|
||||
" c1.append(matpow(b, n))\n",
|
||||
"\n",
|
||||
"with tf.device('/cpu:0'):\n",
|
||||
" sum = tf.add_n(c1) #Addition of all elements in c1, i.e. A^n + B^n\n",
|
||||
"\n",
|
||||
"t1_1 = datetime.datetime.now()\n",
|
||||
"with tf.Session(config=tf.ConfigProto(log_device_placement=log_device_placement)) as sess:\n",
|
||||
" # Runs the op.\n",
|
||||
" sess.run(sum)\n",
|
||||
"t2_1 = datetime.datetime.now()"
|
||||
]
|
||||
},
|
||||
{
|
||||
"cell_type": "code",
|
||||
"execution_count": 7,
|
||||
"metadata": {
|
||||
"collapsed": true
|
||||
},
|
||||
"outputs": [],
|
||||
"source": [
|
||||
"# Multi GPU computing\n",
|
||||
"# GPU:0 computes A^n\n",
|
||||
"with tf.device('/gpu:0'):\n",
|
||||
" #compute A^n and store result in c2\n",
|
||||
" a = tf.constant(A)\n",
|
||||
" c2.append(matpow(a, n))\n",
|
||||
"\n",
|
||||
"#GPU:1 computes B^n\n",
|
||||
"with tf.device('/gpu:1'):\n",
|
||||
" #compute B^n and store result in c2\n",
|
||||
" b = tf.constant(B)\n",
|
||||
" c2.append(matpow(b, n))\n",
|
||||
"\n",
|
||||
"with tf.device('/cpu:0'):\n",
|
||||
" sum = tf.add_n(c2) #Addition of all elements in c2, i.e. A^n + B^n\n",
|
||||
"\n",
|
||||
"t1_2 = datetime.datetime.now()\n",
|
||||
"with tf.Session(config=tf.ConfigProto(log_device_placement=log_device_placement)) as sess:\n",
|
||||
" # Runs the op.\n",
|
||||
" sess.run(sum)\n",
|
||||
"t2_2 = datetime.datetime.now()"
|
||||
]
|
||||
},
|
||||
{
|
||||
"cell_type": "code",
|
||||
"execution_count": 8,
|
||||
"metadata": {
|
||||
"collapsed": false
|
||||
},
|
||||
"outputs": [
|
||||
{
|
||||
"name": "stdout",
|
||||
"output_type": "stream",
|
||||
"text": [
|
||||
"Single GPU computation time: 0:00:11.833497\n",
|
||||
"Multi GPU computation time: 0:00:07.085913\n"
|
||||
]
|
||||
}
|
||||
],
|
||||
"source": [
|
||||
"print \"Single GPU computation time: \" + str(t2_1-t1_1)\n",
|
||||
"print \"Multi GPU computation time: \" + str(t2_2-t1_2)"
|
||||
]
|
||||
}
|
||||
],
|
||||
"metadata": {
|
||||
"kernelspec": {
|
||||
"display_name": "Python 3",
|
||||
"language": "python",
|
||||
"name": "python3"
|
||||
},
|
||||
"language_info": {
|
||||
"codemirror_mode": {
|
||||
"name": "ipython",
|
||||
"version": 3
|
||||
},
|
||||
"file_extension": ".py",
|
||||
"mimetype": "text/x-python",
|
||||
"name": "python",
|
||||
"nbconvert_exporter": "python",
|
||||
"pygments_lexer": "ipython3",
|
||||
"version": "3.4.3"
|
||||
}
|
||||
},
|
||||
"nbformat": 4,
|
||||
"nbformat_minor": 0
|
||||
}
|
||||
File diff suppressed because one or more lines are too long
File diff suppressed because one or more lines are too long
BIN
images/deep.png
Normal file
BIN
images/deep.png
Normal file
Binary file not shown.
|
After Width: | Height: | Size: 27 KiB |
BIN
images/tensorflow.png
Normal file
BIN
images/tensorflow.png
Normal file
Binary file not shown.
|
After Width: | Height: | Size: 8.9 KiB |
BIN
images/theano.png
Normal file
BIN
images/theano.png
Normal file
Binary file not shown.
|
After Width: | Height: | Size: 4.5 KiB |
Loading…
Reference in New Issue
Block a user