mirror of
https://github.com/donnemartin/data-science-ipython-notebooks.git
synced 2024-03-22 13:30:56 +08:00
Merge branch 'feature/deep-learning' into develop
* feature/deep-learning: Add TensorFlow loss viz notebook. Add TensorFlow graph viz notebook. Add TensorFlow multi-gpu computation notebook. Add TensorFlow recurrent neural networks notebook. Add TensorFlow multilayer perceptrons notebook. Add TensorFlow convolutional neural networks notebook. Add TensorFlow AlexNet notebook. Add TensorFlow nearest neighbor notebook. Add TensorFlow logistic regression notebook. Add TensorFlow linear regression notebook. Add TensorFlow basics notebook. Add TensorFlow tutorials supporting material. Break up deep learning section into subsections.
This commit is contained in:
Donne Martin
commit
e3aa2e0122
60
README.md
60
README.md
|
|
@ -85,20 +85,58 @@ IPython Notebook(s) used in [kaggle](https://www.kaggle.com/) competitions and b
|
|||
|
||||
IPython Notebook(s) demonstrating deep learning functionality.
|
||||
|
||||
<br/>
|
||||
<p align="center">
|
||||
<img src="https://avatars0.githubusercontent.com/u/15658638?v=3&s=100">
|
||||
</p>
|
||||
|
||||
### tensor-flow-tutorials
|
||||
|
||||
| Notebook | Description |
|
||||
|--------------------------------------------------------------------------------------------------------------|-------------------------------------------------------------------------------------------------------------------------------------------------------------------|
|
||||
| [tsf-basics](http://nbviewer.ipython.org/github/donnemartin/data-science-ipython-notebooks/blob/master/deep-learning/tensor-flow-examples/1_intro/basic_operations.ipynb) | Learn basic operations in TensorFlow, a library for various kinds of perceptual and language understanding tasks from Google. |
|
||||
| [tsf-linear](http://nbviewer.ipython.org/github/donnemartin/data-science-ipython-notebooks/blob/master/deep-learning/tensor-flow-examples/2_basic_classifiers/linear_regression.ipynb) | Implement linear regression in TensorFlow. |
|
||||
| [tsf-logistic](http://nbviewer.ipython.org/github/donnemartin/data-science-ipython-notebooks/blob/master/deep-learning/tensor-flow-examples/2_basic_classifiers/logistic_regression.ipynb) | Implement logistic regression in TensorFlow. |
|
||||
| [tsf-nn](http://nbviewer.ipython.org/github/donnemartin/data-science-ipython-notebooks/blob/master/deep-learning/tensor-flow-examples/2_basic_classifiers/nearest_neighbor.ipynb) | Implement nearest neighboars in TensorFlow. |
|
||||
| [tsf-alex](http://nbviewer.ipython.org/github/donnemartin/data-science-ipython-notebooks/blob/master/deep-learning/tensor-flow-examples/3_neural_networks/alexnet.ipynb) | Implement AlexNet in TensorFlow. |
|
||||
| [tsf-cnn](http://nbviewer.ipython.org/github/donnemartin/data-science-ipython-notebooks/blob/master/deep-learning/tensor-flow-examples/3_neural_networks/convolutional_network.ipynb) | Implement convolutional neural networks in TensorFlow. |
|
||||
| [tsf-mlp](http://nbviewer.ipython.org/github/donnemartin/data-science-ipython-notebooks/blob/master/deep-learning/tensor-flow-examples/3_neural_networks/multilayer_perceptron.ipynb) | Implement multilayer perceptrons in TensorFlow. |
|
||||
| [tsf-rnn](http://nbviewer.ipython.org/github/donnemartin/data-science-ipython-notebooks/blob/master/deep-learning/tensor-flow-examples/3_neural_networks/recurrent_network.ipynb) | Implement recurrent neural networks in TensorFlow. |
|
||||
| [tsf-gpu](http://nbviewer.ipython.org/github/donnemartin/data-science-ipython-notebooks/blob/master/deep-learning/tensor-flow-examples/4_multi_gpu/multigpu_basics.ipynb) | Learn about basic multi-GPU computation in TensorFlow. |
|
||||
| [tsf-gviz](http://nbviewer.ipython.org/github/donnemartin/data-science-ipython-notebooks/blob/master/deep-learning/tensor-flow-examples/5_ui/graph_visualization.ipynb) | Learn about graph visualization in TensorFlow. |
|
||||
| [tsf-lviz](http://nbviewer.ipython.org/github/donnemartin/data-science-ipython-notebooks/blob/master/deep-learning/tensor-flow-examples/5_ui/loss_visualization.ipynb) | Learn about loss visualization in TensorFlow. |
|
||||
|
||||
### tensor-flow-exercises
|
||||
|
||||
| Notebook | Description |
|
||||
|--------------------------------------------------------------------------------------------------------------|-------------------------------------------------------------------------------------------------------------------------------------------------------------------|
|
||||
| [tsf-not-mnist](http://nbviewer.ipython.org/github/donnemartin/data-science-ipython-notebooks/blob/master/deep-learning/tensor-flow-exercises/1_notmnist.ipynb) | Learn simple data curation by creating a pickle with formatted datasets for training, development and testing in TensorFlow. |
|
||||
| [tsf-fully-connected](http://nbviewer.ipython.org/github/donnemartin/data-science-ipython-notebooks/blob/master/deep-learning/tensor-flow-exercises/2_fullyconnected.ipynb) | Progressively train deeper and more accurate models using logistic regression and neural networks in TensorFlow. |
|
||||
| [tsf-regularization](http://nbviewer.ipython.org/github/donnemartin/data-science-ipython-notebooks/blob/master/deep-learning/tensor-flow-exercises/3_regularization.ipynb) | Explore regularization techniques by training fully connected networks to classify notMNIST characters in TensorFlow. |
|
||||
| [tsf-convolutions](http://nbviewer.ipython.org/github/donnemartin/data-science-ipython-notebooks/blob/master/deep-learning/tensor-flow-exercises/4_convolutions.ipynb) | Create convolutional neural networks in TensorFlow. |
|
||||
| [tsf-word2vec](http://nbviewer.ipython.org/github/donnemartin/data-science-ipython-notebooks/blob/master/deep-learning/tensor-flow-exercises/5_word2vec.ipynb) | Train a skip-gram model over Text8 data in TensorFlow. |
|
||||
| [tsf-lstm](http://nbviewer.ipython.org/github/donnemartin/data-science-ipython-notebooks/blob/master/deep-learning/tensor-flow-exercises/6_lstm.ipynb) | Train a LSTM character model over Text8 data in TensorFlow. |
|
||||
| [theano-intro](http://nbviewer.ipython.org/github/donnemartin/data-science-ipython-notebooks/blob/master/deep-learning/theano-tutorial/intro_theano/intro_theano.ipynb) | Intro to Theano, which allows you to define, optimize, and evaluate mathematical expressions involving multi-dimensional arrays efficiently. It can use GPUs and perform efficient symbolic differentiation. |
|
||||
| [theano-scan](http://nbviewer.ipython.org/github/donnemartin/data-science-ipython-notebooks/blob/master/deep-learning/theano-tutorial/scan_tutorial/scan_tutorial.ipynb) | Learn scans, a mechanism to perform loops in a Theano graph. |
|
||||
| [theano-logistic](http://nbviewer.ipython.org/github/donnemartin/data-science-ipython-notebooks/blob/master/deep-learning/theano-tutorial/intro_theano/logistic_regression.ipynb) | Implement logistic regression in Theano. |
|
||||
| [theano-rnn](http://nbviewer.ipython.org/github/donnemartin/data-science-ipython-notebooks/blob/master/deep-learning/theano-tutorial/rnn_tutorial/simple_rnn.ipynb) | Implement recurrent neural networks in Theano. |
|
||||
| [theano-mlp](http://nbviewer.ipython.org/github/donnemartin/data-science-ipython-notebooks/blob/master/deep-learning/theano-tutorial/theano_mlp/theano_mlp.ipynb) | Implement multilayer perceptrons in Theano. |
|
||||
| [deep-dream](http://nbviewer.ipython.org/github/donnemartin/data-science-ipython-notebooks/blob/master/deep-learning/deep-dream/dream.ipynb) | Caffe-based computer vision program which uses a convolutional neural network to find and enhance patterns in images. |
|
||||
| [tsf-fully-connected](http://nbviewer.ipython.org/github/donnemartin/data-science-ipython-notebooks/blob/master/deep-learning/tensor-flow-exercises/2_fullyconnected.ipynb) | Progressively train deeper and more accurate models using logistic regression and neural networks in TensorFlow. |
|
||||
| [tsf-regularization](http://nbviewer.ipython.org/github/donnemartin/data-science-ipython-notebooks/blob/master/deep-learning/tensor-flow-exercises/3_regularization.ipynb) | Explore regularization techniques by training fully connected networks to classify notMNIST characters in TensorFlow. |
|
||||
| [tsf-convolutions](http://nbviewer.ipython.org/github/donnemartin/data-science-ipython-notebooks/blob/master/deep-learning/tensor-flow-exercises/4_convolutions.ipynb) | Create convolutional neural networks in TensorFlow. |
|
||||
| [tsf-word2vec](http://nbviewer.ipython.org/github/donnemartin/data-science-ipython-notebooks/blob/master/deep-learning/tensor-flow-exercises/5_word2vec.ipynb) | Train a skip-gram model over Text8 data in TensorFlow. |
|
||||
| [tsf-lstm](http://nbviewer.ipython.org/github/donnemartin/data-science-ipython-notebooks/blob/master/deep-learning/tensor-flow-exercises/6_lstm.ipynb) | Train a LSTM character model over Text8 data in TensorFlow. |
|
||||
|
||||
<br/>
|
||||
<p align="center">
|
||||
<img src="http://www.deeplearning.net/software/theano/_static/theano_logo_allblue_200x46.png">
|
||||
</p>
|
||||
|
||||
### theano-tutorials
|
||||
|
||||
| Notebook | Description |
|
||||
|--------------------------------------------------------------------------------------------------------------|-------------------------------------------------------------------------------------------------------------------------------------------------------------------|
|
||||
| [theano-intro](http://nbviewer.ipython.org/github/donnemartin/data-science-ipython-notebooks/blob/master/deep-learning/theano-tutorial/intro_theano/intro_theano.ipynb) | Intro to Theano, which allows you to define, optimize, and evaluate mathematical expressions involving multi-dimensional arrays efficiently. It can use GPUs and perform efficient symbolic differentiation. |
|
||||
| [theano-scan](http://nbviewer.ipython.org/github/donnemartin/data-science-ipython-notebooks/blob/master/deep-learning/theano-tutorial/scan_tutorial/scan_tutorial.ipynb) | Learn scans, a mechanism to perform loops in a Theano graph. |
|
||||
| [theano-logistic](http://nbviewer.ipython.org/github/donnemartin/data-science-ipython-notebooks/blob/master/deep-learning/theano-tutorial/intro_theano/logistic_regression.ipynb) | Implement logistic regression in Theano. |
|
||||
| [theano-rnn](http://nbviewer.ipython.org/github/donnemartin/data-science-ipython-notebooks/blob/master/deep-learning/theano-tutorial/rnn_tutorial/simple_rnn.ipynb) | Implement recurrent neural networks in Theano. |
|
||||
| [theano-mlp](http://nbviewer.ipython.org/github/donnemartin/data-science-ipython-notebooks/blob/master/deep-learning/theano-tutorial/theano_mlp/theano_mlp.ipynb) | Implement multilayer perceptrons in Theano. |
|
||||
|
||||
### deep-learning-misc
|
||||
|
||||
| Notebook | Description |
|
||||
|--------------------------------------------------------------------------------------------------------------|-------------------------------------------------------------------------------------------------------------------------------------------------------------------|
|
||||
| [deep-dream](http://nbviewer.ipython.org/github/donnemartin/data-science-ipython-notebooks/blob/master/deep-learning/deep-dream/dream.ipynb) | Caffe-based computer vision program which uses a convolutional neural network to find and enhance patterns in images. |
|
||||
|
||||
<br/>
|
||||
<p align="center">
|
||||
|
|
|
|||
486
deep-learning/tensor-flow-examples/Setup_TensorFlow.md
Normal file
486
deep-learning/tensor-flow-examples/Setup_TensorFlow.md
Normal file
|
|
@ -0,0 +1,486 @@
|
|||
_From TensorFlow Official docs_
|
||||
|
||||
# Download and Setup <a class="md-anchor" id="AUTOGENERATED-download-and-setup"></a>
|
||||
|
||||
You can install TensorFlow using our provided binary packages or from source.
|
||||
|
||||
## Binary Installation <a class="md-anchor" id="AUTOGENERATED-binary-installation"></a>
|
||||
|
||||
The TensorFlow Python API currently requires Python 2.7: we are
|
||||
[working](https://github.com/tensorflow/tensorflow/issues/1) on adding support
|
||||
for Python 3.
|
||||
|
||||
The simplest way to install TensorFlow is using
|
||||
[pip](https://pypi.python.org/pypi/pip) for both Linux and Mac.
|
||||
|
||||
If you encounter installation errors, see
|
||||
[common problems](#common_install_problems) for some solutions. To simplify
|
||||
installation, please consider using our virtualenv-based instructions
|
||||
[here](#virtualenv_install).
|
||||
|
||||
### Ubuntu/Linux 64-bit <a class="md-anchor" id="AUTOGENERATED-ubuntu-linux-64-bit"></a>
|
||||
|
||||
```bash
|
||||
# For CPU-only version
|
||||
$ pip install https://storage.googleapis.com/tensorflow/linux/cpu/tensorflow-0.5.0-cp27-none-linux_x86_64.whl
|
||||
|
||||
# For GPU-enabled version (only install this version if you have the CUDA sdk installed)
|
||||
$ pip install https://storage.googleapis.com/tensorflow/linux/gpu/tensorflow-0.5.0-cp27-none-linux_x86_64.whl
|
||||
```
|
||||
|
||||
### Mac OS X <a class="md-anchor" id="AUTOGENERATED-mac-os-x"></a>
|
||||
|
||||
On OS X, we recommend installing [homebrew](http://brew.sh) and `brew install
|
||||
python` before proceeding, or installing TensorFlow within [virtualenv](#virtualenv_install).
|
||||
|
||||
```bash
|
||||
# Only CPU-version is available at the moment.
|
||||
$ pip install https://storage.googleapis.com/tensorflow/mac/tensorflow-0.5.0-py2-none-any.whl
|
||||
```
|
||||
|
||||
## Docker-based installation <a class="md-anchor" id="AUTOGENERATED-docker-based-installation"></a>
|
||||
|
||||
We also support running TensorFlow via [Docker](http://docker.com/), which lets
|
||||
you avoid worrying about setting up dependencies.
|
||||
|
||||
First, [install Docker](http://docs.docker.com/engine/installation/). Once
|
||||
Docker is up and running, you can start a container with one command:
|
||||
|
||||
```bash
|
||||
$ docker run -it b.gcr.io/tensorflow/tensorflow
|
||||
```
|
||||
|
||||
This will start a container with TensorFlow and all its dependencies already
|
||||
installed.
|
||||
|
||||
### Additional images <a class="md-anchor" id="AUTOGENERATED-additional-images"></a>
|
||||
|
||||
The default Docker image above contains just a minimal set of libraries for
|
||||
getting up and running with TensorFlow. We also have the following container,
|
||||
which you can use in the `docker run` command above:
|
||||
|
||||
* `b.gcr.io/tensorflow/tensorflow-full`: Contains a complete TensorFlow source
|
||||
installation, including all utilities needed to build and run TensorFlow. This
|
||||
makes it easy to experiment directly with the source, without needing to
|
||||
install any of the dependencies described above.
|
||||
|
||||
## VirtualEnv-based installation <a class="md-anchor" id="virtualenv_install"></a>
|
||||
|
||||
We recommend using [virtualenv](https://pypi.python.org/pypi/virtualenv) to
|
||||
create an isolated container and install TensorFlow in that container -- it is
|
||||
optional but makes verifying installation issues easier.
|
||||
|
||||
First, install all required tools:
|
||||
|
||||
```bash
|
||||
# On Linux:
|
||||
$ sudo apt-get install python-pip python-dev python-virtualenv
|
||||
|
||||
# On Mac:
|
||||
$ sudo easy_install pip # If pip is not already installed
|
||||
$ sudo pip install --upgrade virtualenv
|
||||
```
|
||||
|
||||
Next, set up a new virtualenv environment. To set it up in the
|
||||
directory `~/tensorflow`, run:
|
||||
|
||||
```bash
|
||||
$ virtualenv --system-site-packages ~/tensorflow
|
||||
$ cd ~/tensorflow
|
||||
```
|
||||
|
||||
Then activate the virtualenv:
|
||||
|
||||
```bash
|
||||
$ source bin/activate # If using bash
|
||||
$ source bin/activate.csh # If using csh
|
||||
(tensorflow)$ # Your prompt should change
|
||||
```
|
||||
|
||||
Inside the virtualenv, install TensorFlow:
|
||||
|
||||
```bash
|
||||
# For CPU-only linux x86_64 version
|
||||
(tensorflow)$ pip install --upgrade https://storage.googleapis.com/tensorflow/linux/cpu/tensorflow-0.5.0-cp27-none-linux_x86_64.whl
|
||||
|
||||
# For GPU-enabled linux x86_64 version
|
||||
(tensorflow)$ pip install --upgrade https://storage.googleapis.com/tensorflow/linux/gpu/tensorflow-0.5.0-cp27-none-linux_x86_64.whl
|
||||
|
||||
# For Mac CPU-only version
|
||||
(tensorflow)$ pip install --upgrade https://storage.googleapis.com/tensorflow/mac/tensorflow-0.5.0-py2-none-any.whl
|
||||
```
|
||||
|
||||
Make sure you have downloaded the source code for TensorFlow, and then you can
|
||||
then run an example TensorFlow program like:
|
||||
|
||||
```bash
|
||||
(tensorflow)$ cd tensorflow/models/image/mnist
|
||||
(tensorflow)$ python convolutional.py
|
||||
|
||||
# When you are done using TensorFlow:
|
||||
(tensorflow)$ deactivate # Deactivate the virtualenv
|
||||
|
||||
$ # Your prompt should change back
|
||||
```
|
||||
|
||||
## Try your first TensorFlow program <a class="md-anchor" id="AUTOGENERATED-try-your-first-tensorflow-program"></a>
|
||||
|
||||
### (Optional) Enable GPU Support <a class="md-anchor" id="AUTOGENERATED--optional--enable-gpu-support"></a>
|
||||
|
||||
If you installed the GPU-enabled TensorFlow pip binary, you must have the
|
||||
correct versions of the CUDA SDK and CUDNN installed on your
|
||||
system. Please see [the CUDA installation instructions](#install_cuda).
|
||||
|
||||
You also need to set the `LD_LIBRARY_PATH` and `CUDA_HOME` environment
|
||||
variables. Consider adding the commands below to your `~/.bash_profile`. These
|
||||
assume your CUDA installation is in `/usr/local/cuda`:
|
||||
|
||||
```bash
|
||||
export LD_LIBRARY_PATH="$LD_LIBRARY_PATH:/usr/local/cuda/lib64"
|
||||
export CUDA_HOME=/usr/local/cuda
|
||||
```
|
||||
|
||||
### Run TensorFlow <a class="md-anchor" id="AUTOGENERATED-run-tensorflow"></a>
|
||||
|
||||
Open a python terminal:
|
||||
|
||||
```bash
|
||||
$ python
|
||||
|
||||
>>> import tensorflow as tf
|
||||
>>> hello = tf.constant('Hello, TensorFlow!')
|
||||
>>> sess = tf.Session()
|
||||
>>> print sess.run(hello)
|
||||
Hello, TensorFlow!
|
||||
>>> a = tf.constant(10)
|
||||
>>> b = tf.constant(32)
|
||||
>>> print sess.run(a+b)
|
||||
42
|
||||
>>>
|
||||
|
||||
```
|
||||
|
||||
## Installing from sources <a class="md-anchor" id="source"></a>
|
||||
|
||||
### Clone the TensorFlow repository <a class="md-anchor" id="AUTOGENERATED-clone-the-tensorflow-repository"></a>
|
||||
|
||||
```bash
|
||||
$ git clone --recurse-submodules https://github.com/tensorflow/tensorflow
|
||||
```
|
||||
|
||||
`--recurse-submodules` is required to fetch the protobuf library that TensorFlow
|
||||
depends on.
|
||||
|
||||
### Installation for Linux <a class="md-anchor" id="AUTOGENERATED-installation-for-linux"></a>
|
||||
|
||||
#### Install Bazel <a class="md-anchor" id="AUTOGENERATED-install-bazel"></a>
|
||||
|
||||
|
||||
Follow instructions [here](http://bazel.io/docs/install.html) to install the
|
||||
dependencies for Bazel. Then download bazel version 0.1.1 using the
|
||||
[installer for your system](https://github.com/bazelbuild/bazel/releases) and
|
||||
run the installer as mentioned there:
|
||||
|
||||
```bash
|
||||
$ chmod +x PATH_TO_INSTALL.SH
|
||||
$ ./PATH_TO_INSTALL.SH --user
|
||||
```
|
||||
|
||||
Remember to replace `PATH_TO_INSTALL.SH` to point to the location where you
|
||||
downloaded the installer.
|
||||
|
||||
Finally, follow the instructions in that script to place bazel into your binary
|
||||
path.
|
||||
|
||||
#### Install other dependencies <a class="md-anchor" id="AUTOGENERATED-install-other-dependencies"></a>
|
||||
|
||||
```bash
|
||||
$ sudo apt-get install python-numpy swig python-dev
|
||||
```
|
||||
|
||||
#### Optional: Install CUDA (GPUs on Linux) <a class="md-anchor" id="install_cuda"></a>
|
||||
|
||||
In order to build or run TensorFlow with GPU support, both Cuda Toolkit 7.0 and
|
||||
CUDNN 6.5 V2 from NVIDIA need to be installed.
|
||||
|
||||
TensorFlow GPU support requires having a GPU card with NVidia Compute Capability >= 3.5. Supported cards include but are not limited to:
|
||||
|
||||
* NVidia Titan
|
||||
* NVidia Titan X
|
||||
* NVidia K20
|
||||
* NVidia K40
|
||||
|
||||
##### Download and install Cuda Toolkit 7.0 <a class="md-anchor" id="AUTOGENERATED-download-and-install-cuda-toolkit-7.0"></a>
|
||||
|
||||
https://developer.nvidia.com/cuda-toolkit-70
|
||||
|
||||
Install the toolkit into e.g. `/usr/local/cuda`
|
||||
|
||||
##### Download and install CUDNN Toolkit 6.5 <a class="md-anchor" id="AUTOGENERATED-download-and-install-cudnn-toolkit-6.5"></a>
|
||||
|
||||
https://developer.nvidia.com/rdp/cudnn-archive
|
||||
|
||||
Uncompress and copy the cudnn files into the toolkit directory. Assuming the
|
||||
toolkit is installed in `/usr/local/cuda`:
|
||||
|
||||
``` bash
|
||||
tar xvzf cudnn-6.5-linux-x64-v2.tgz
|
||||
sudo cp cudnn-6.5-linux-x64-v2/cudnn.h /usr/local/cuda/include
|
||||
sudo cp cudnn-6.5-linux-x64-v2/libcudnn* /usr/local/cuda/lib64
|
||||
```
|
||||
|
||||
##### Configure TensorFlow's canonical view of Cuda libraries <a class="md-anchor" id="AUTOGENERATED-configure-tensorflow-s-canonical-view-of-cuda-libraries"></a>
|
||||
From the root of your source tree, run:
|
||||
|
||||
``` bash
|
||||
$ ./configure
|
||||
Do you wish to build TensorFlow with GPU support? [y/n] y
|
||||
GPU support will be enabled for TensorFlow
|
||||
|
||||
Please specify the location where CUDA 7.0 toolkit is installed. Refer to
|
||||
README.md for more details. [default is: /usr/local/cuda]: /usr/local/cuda
|
||||
|
||||
Please specify the location where CUDNN 6.5 V2 library is installed. Refer to
|
||||
README.md for more details. [default is: /usr/local/cuda]: /usr/local/cuda
|
||||
|
||||
Setting up Cuda include
|
||||
Setting up Cuda lib64
|
||||
Setting up Cuda bin
|
||||
Setting up Cuda nvvm
|
||||
Configuration finished
|
||||
```
|
||||
|
||||
This creates a canonical set of symbolic links to the Cuda libraries on your system.
|
||||
Every time you change the Cuda library paths you need to run this step again before
|
||||
you invoke the bazel build command.
|
||||
|
||||
##### Build your target with GPU support. <a class="md-anchor" id="AUTOGENERATED-build-your-target-with-gpu-support."></a>
|
||||
From the root of your source tree, run:
|
||||
|
||||
```bash
|
||||
$ bazel build -c opt --config=cuda //tensorflow/cc:tutorials_example_trainer
|
||||
|
||||
$ bazel-bin/tensorflow/cc/tutorials_example_trainer --use_gpu
|
||||
# Lots of output. This tutorial iteratively calculates the major eigenvalue of
|
||||
# a 2x2 matrix, on GPU. The last few lines look like this.
|
||||
000009/000005 lambda = 2.000000 x = [0.894427 -0.447214] y = [1.788854 -0.894427]
|
||||
000006/000001 lambda = 2.000000 x = [0.894427 -0.447214] y = [1.788854 -0.894427]
|
||||
000009/000009 lambda = 2.000000 x = [0.894427 -0.447214] y = [1.788854 -0.894427]
|
||||
```
|
||||
|
||||
Note that "--config=cuda" is needed to enable the GPU support.
|
||||
|
||||
##### Enabling Cuda 3.0. <a class="md-anchor" id="AUTOGENERATED-enabling-cuda-3.0."></a>
|
||||
TensorFlow officially supports Cuda devices with 3.5 and 5.2 compute
|
||||
capabilities. In order to enable earlier Cuda devices such as Grid K520, you
|
||||
need to target Cuda 3.0. This can be done through TensorFlow unofficial
|
||||
settings with "configure".
|
||||
|
||||
```bash
|
||||
$ TF_UNOFFICIAL_SETTING=1 ./configure
|
||||
|
||||
# Same as the official settings above
|
||||
|
||||
WARNING: You are configuring unofficial settings in TensorFlow. Because some
|
||||
external libraries are not backward compatible, these settings are largely
|
||||
untested and unsupported.
|
||||
|
||||
Please specify a list of comma-separated Cuda compute capabilities you want to
|
||||
build with. You can find the compute capability of your device at:
|
||||
https://developer.nvidia.com/cuda-gpus.
|
||||
Please note that each additional compute capability significantly increases
|
||||
your build time and binary size. [Default is: "3.5,5.2"]: 3.0
|
||||
|
||||
Setting up Cuda include
|
||||
Setting up Cuda lib64
|
||||
Setting up Cuda bin
|
||||
Setting up Cuda nvvm
|
||||
Configuration finished
|
||||
```
|
||||
|
||||
##### Known issues <a class="md-anchor" id="AUTOGENERATED-known-issues"></a>
|
||||
|
||||
* Although it is possible to build both Cuda and non-Cuda configs under the same
|
||||
source tree, we recommend to run "bazel clean" when switching between these two
|
||||
configs in the same source tree.
|
||||
|
||||
* You have to run configure before running bazel build. Otherwise, the build
|
||||
will fail with a clear error message. In the future, we might consider making
|
||||
this more conveninent by including the configure step in our build process,
|
||||
given necessary bazel new feature support.
|
||||
|
||||
### Installation for Mac OS X <a class="md-anchor" id="AUTOGENERATED-installation-for-mac-os-x"></a>
|
||||
|
||||
Mac needs the same set of dependencies as Linux, however their installing those
|
||||
dependencies is different. Here is a set of useful links to help with installing
|
||||
the dependencies on Mac OS X :
|
||||
|
||||
#### Bazel <a class="md-anchor" id="AUTOGENERATED-bazel"></a>
|
||||
|
||||
Look for installation instructions for Mac OS X on
|
||||
[this](http://bazel.io/docs/install.html) page.
|
||||
|
||||
#### SWIG <a class="md-anchor" id="AUTOGENERATED-swig"></a>
|
||||
|
||||
[Mac OS X installation](http://www.swig.org/Doc3.0/Preface.html#Preface_osx_installation).
|
||||
|
||||
Notes : You need to install
|
||||
[PCRE](ftp://ftp.csx.cam.ac.uk/pub/software/programming/pcre/) and *NOT* PCRE2.
|
||||
|
||||
#### Numpy <a class="md-anchor" id="AUTOGENERATED-numpy"></a>
|
||||
|
||||
Follow installation instructions [here](http://docs.scipy.org/doc/numpy/user/install.html).
|
||||
|
||||
|
||||
### Create the pip package and install <a class="md-anchor" id="create-pip"></a>
|
||||
|
||||
```bash
|
||||
$ bazel build -c opt //tensorflow/tools/pip_package:build_pip_package
|
||||
|
||||
# To build with GPU support:
|
||||
$ bazel build -c opt --config=cuda //tensorflow/tools/pip_package:build_pip_package
|
||||
|
||||
$ bazel-bin/tensorflow/tools/pip_package/build_pip_package /tmp/tensorflow_pkg
|
||||
|
||||
# The name of the .whl file will depend on your platform.
|
||||
$ pip install /tmp/tensorflow_pkg/tensorflow-0.5.0-cp27-none-linux_x86_64.whl
|
||||
```
|
||||
|
||||
## Train your first TensorFlow neural net model <a class="md-anchor" id="AUTOGENERATED-train-your-first-tensorflow-neural-net-model"></a>
|
||||
|
||||
Starting from the root of your source tree, run:
|
||||
|
||||
```python
|
||||
$ cd tensorflow/models/image/mnist
|
||||
$ python convolutional.py
|
||||
Succesfully downloaded train-images-idx3-ubyte.gz 9912422 bytes.
|
||||
Succesfully downloaded train-labels-idx1-ubyte.gz 28881 bytes.
|
||||
Succesfully downloaded t10k-images-idx3-ubyte.gz 1648877 bytes.
|
||||
Succesfully downloaded t10k-labels-idx1-ubyte.gz 4542 bytes.
|
||||
Extracting data/train-images-idx3-ubyte.gz
|
||||
Extracting data/train-labels-idx1-ubyte.gz
|
||||
Extracting data/t10k-images-idx3-ubyte.gz
|
||||
Extracting data/t10k-labels-idx1-ubyte.gz
|
||||
Initialized!
|
||||
Epoch 0.00
|
||||
Minibatch loss: 12.054, learning rate: 0.010000
|
||||
Minibatch error: 90.6%
|
||||
Validation error: 84.6%
|
||||
Epoch 0.12
|
||||
Minibatch loss: 3.285, learning rate: 0.010000
|
||||
Minibatch error: 6.2%
|
||||
Validation error: 7.0%
|
||||
...
|
||||
...
|
||||
```
|
||||
|
||||
## Common Problems <a class="md-anchor" id="common_install_problems"></a>
|
||||
|
||||
### GPU-related issues <a class="md-anchor" id="AUTOGENERATED-gpu-related-issues"></a>
|
||||
|
||||
If you encounter the following when trying to run a TensorFlow program:
|
||||
|
||||
```python
|
||||
ImportError: libcudart.so.7.0: cannot open shared object file: No such file or directory
|
||||
```
|
||||
|
||||
Make sure you followed the the GPU installation [instructions](#install_cuda).
|
||||
|
||||
### Pip installation issues <a class="md-anchor" id="AUTOGENERATED-pip-installation-issues"></a>
|
||||
|
||||
#### Can't find setup.py <a class="md-anchor" id="AUTOGENERATED-can-t-find-setup.py"></a>
|
||||
|
||||
If, during `pip install`, you encounter an error like:
|
||||
|
||||
```bash
|
||||
...
|
||||
IOError: [Errno 2] No such file or directory: '/tmp/pip-o6Tpui-build/setup.py'
|
||||
```
|
||||
|
||||
Solution: upgrade your version of `pip`:
|
||||
|
||||
```bash
|
||||
pip install --upgrade pip
|
||||
```
|
||||
|
||||
This may require `sudo`, depending on how `pip` is installed.
|
||||
|
||||
#### SSLError: SSL_VERIFY_FAILED <a class="md-anchor" id="AUTOGENERATED-sslerror--ssl_verify_failed"></a>
|
||||
|
||||
If, during pip install from a URL, you encounter an error like:
|
||||
|
||||
```bash
|
||||
...
|
||||
SSLError: [SSL: CERTIFICATE_VERIFY_FAILED] certificate verify failed
|
||||
```
|
||||
|
||||
Solution: Download the wheel manually via curl or wget, and pip install locally.
|
||||
|
||||
### On Linux <a class="md-anchor" id="AUTOGENERATED-on-linux"></a>
|
||||
|
||||
If you encounter:
|
||||
|
||||
```python
|
||||
...
|
||||
"__add__", "__radd__",
|
||||
^
|
||||
SyntaxError: invalid syntax
|
||||
```
|
||||
|
||||
Solution: make sure you are using Python 2.7.
|
||||
|
||||
### On MacOSX <a class="md-anchor" id="AUTOGENERATED-on-macosx"></a>
|
||||
|
||||
|
||||
If you encounter:
|
||||
|
||||
```python
|
||||
import six.moves.copyreg as copyreg
|
||||
|
||||
ImportError: No module named copyreg
|
||||
```
|
||||
|
||||
Solution: TensorFlow depends on protobuf, which requires `six-1.10.0`. Apple's
|
||||
default python environment has `six-1.4.1` and may be difficult to upgrade.
|
||||
There are several ways to fix this:
|
||||
|
||||
1. Upgrade the system-wide copy of `six`:
|
||||
|
||||
```bash
|
||||
sudo easy_install -U six
|
||||
```
|
||||
|
||||
2. Install a separate copy of python via homebrew:
|
||||
|
||||
```bash
|
||||
brew install python
|
||||
```
|
||||
|
||||
3. Build or use TensorFlow
|
||||
[within `virtualenv`](#virtualenv_install).
|
||||
|
||||
|
||||
|
||||
If you encounter:
|
||||
|
||||
```
|
||||
>>> import tensorflow as tf
|
||||
Traceback (most recent call last):
|
||||
File "<stdin>", line 1, in <module>
|
||||
File "/usr/local/lib/python2.7/site-packages/tensorflow/__init__.py", line 4, in <module>
|
||||
from tensorflow.python import *
|
||||
File "/usr/local/lib/python2.7/site-packages/tensorflow/python/__init__.py", line 13, in <module>
|
||||
from tensorflow.core.framework.graph_pb2 import *
|
||||
...
|
||||
File "/usr/local/lib/python2.7/site-packages/tensorflow/core/framework/tensor_shape_pb2.py", line 22, in <module>
|
||||
serialized_pb=_b('\n,tensorflow/core/framework/tensor_shape.proto\x12\ntensorflow\"d\n\x10TensorShapeProto\x12-\n\x03\x64im\x18\x02 \x03(\x0b\x32 .tensorflow.TensorShapeProto.Dim\x1a!\n\x03\x44im\x12\x0c\n\x04size\x18\x01 \x01(\x03\x12\x0c\n\x04name\x18\x02 \x01(\tb\x06proto3')
|
||||
TypeError: __init__() got an unexpected keyword argument 'syntax'
|
||||
```
|
||||
|
||||
This is due to a conflict between protobuf versions (we require protobuf 3.0.0).
|
||||
The best current solution is to make sure older versions of protobuf are not
|
||||
installed, such as:
|
||||
|
||||
```bash
|
||||
brew reinstall --devel protobuf
|
||||
```
|
||||
144
deep-learning/tensor-flow-examples/input_data.py
Normal file
144
deep-learning/tensor-flow-examples/input_data.py
Normal file
|
|
@ -0,0 +1,144 @@
|
|||
"""Functions for downloading and reading MNIST data."""
|
||||
from __future__ import print_function
|
||||
import gzip
|
||||
import os
|
||||
import urllib
|
||||
import numpy
|
||||
SOURCE_URL = 'http://yann.lecun.com/exdb/mnist/'
|
||||
def maybe_download(filename, work_directory):
|
||||
"""Download the data from Yann's website, unless it's already here."""
|
||||
if not os.path.exists(work_directory):
|
||||
os.mkdir(work_directory)
|
||||
filepath = os.path.join(work_directory, filename)
|
||||
if not os.path.exists(filepath):
|
||||
filepath, _ = urllib.urlretrieve(SOURCE_URL + filename, filepath)
|
||||
statinfo = os.stat(filepath)
|
||||
print('Succesfully downloaded', filename, statinfo.st_size, 'bytes.')
|
||||
return filepath
|
||||
def _read32(bytestream):
|
||||
dt = numpy.dtype(numpy.uint32).newbyteorder('>')
|
||||
return numpy.frombuffer(bytestream.read(4), dtype=dt)
|
||||
def extract_images(filename):
|
||||
"""Extract the images into a 4D uint8 numpy array [index, y, x, depth]."""
|
||||
print('Extracting', filename)
|
||||
with gzip.open(filename) as bytestream:
|
||||
magic = _read32(bytestream)
|
||||
if magic != 2051:
|
||||
raise ValueError(
|
||||
'Invalid magic number %d in MNIST image file: %s' %
|
||||
(magic, filename))
|
||||
num_images = _read32(bytestream)
|
||||
rows = _read32(bytestream)
|
||||
cols = _read32(bytestream)
|
||||
buf = bytestream.read(rows * cols * num_images)
|
||||
data = numpy.frombuffer(buf, dtype=numpy.uint8)
|
||||
data = data.reshape(num_images, rows, cols, 1)
|
||||
return data
|
||||
def dense_to_one_hot(labels_dense, num_classes=10):
|
||||
"""Convert class labels from scalars to one-hot vectors."""
|
||||
num_labels = labels_dense.shape[0]
|
||||
index_offset = numpy.arange(num_labels) * num_classes
|
||||
labels_one_hot = numpy.zeros((num_labels, num_classes))
|
||||
labels_one_hot.flat[index_offset + labels_dense.ravel()] = 1
|
||||
return labels_one_hot
|
||||
def extract_labels(filename, one_hot=False):
|
||||
"""Extract the labels into a 1D uint8 numpy array [index]."""
|
||||
print('Extracting', filename)
|
||||
with gzip.open(filename) as bytestream:
|
||||
magic = _read32(bytestream)
|
||||
if magic != 2049:
|
||||
raise ValueError(
|
||||
'Invalid magic number %d in MNIST label file: %s' %
|
||||
(magic, filename))
|
||||
num_items = _read32(bytestream)
|
||||
buf = bytestream.read(num_items)
|
||||
labels = numpy.frombuffer(buf, dtype=numpy.uint8)
|
||||
if one_hot:
|
||||
return dense_to_one_hot(labels)
|
||||
return labels
|
||||
class DataSet(object):
|
||||
def __init__(self, images, labels, fake_data=False):
|
||||
if fake_data:
|
||||
self._num_examples = 10000
|
||||
else:
|
||||
assert images.shape[0] == labels.shape[0], (
|
||||
"images.shape: %s labels.shape: %s" % (images.shape,
|
||||
labels.shape))
|
||||
self._num_examples = images.shape[0]
|
||||
# Convert shape from [num examples, rows, columns, depth]
|
||||
# to [num examples, rows*columns] (assuming depth == 1)
|
||||
assert images.shape[3] == 1
|
||||
images = images.reshape(images.shape[0],
|
||||
images.shape[1] * images.shape[2])
|
||||
# Convert from [0, 255] -> [0.0, 1.0].
|
||||
images = images.astype(numpy.float32)
|
||||
images = numpy.multiply(images, 1.0 / 255.0)
|
||||
self._images = images
|
||||
self._labels = labels
|
||||
self._epochs_completed = 0
|
||||
self._index_in_epoch = 0
|
||||
@property
|
||||
def images(self):
|
||||
return self._images
|
||||
@property
|
||||
def labels(self):
|
||||
return self._labels
|
||||
@property
|
||||
def num_examples(self):
|
||||
return self._num_examples
|
||||
@property
|
||||
def epochs_completed(self):
|
||||
return self._epochs_completed
|
||||
def next_batch(self, batch_size, fake_data=False):
|
||||
"""Return the next `batch_size` examples from this data set."""
|
||||
if fake_data:
|
||||
fake_image = [1.0 for _ in xrange(784)]
|
||||
fake_label = 0
|
||||
return [fake_image for _ in xrange(batch_size)], [
|
||||
fake_label for _ in xrange(batch_size)]
|
||||
start = self._index_in_epoch
|
||||
self._index_in_epoch += batch_size
|
||||
if self._index_in_epoch > self._num_examples:
|
||||
# Finished epoch
|
||||
self._epochs_completed += 1
|
||||
# Shuffle the data
|
||||
perm = numpy.arange(self._num_examples)
|
||||
numpy.random.shuffle(perm)
|
||||
self._images = self._images[perm]
|
||||
self._labels = self._labels[perm]
|
||||
# Start next epoch
|
||||
start = 0
|
||||
self._index_in_epoch = batch_size
|
||||
assert batch_size <= self._num_examples
|
||||
end = self._index_in_epoch
|
||||
return self._images[start:end], self._labels[start:end]
|
||||
def read_data_sets(train_dir, fake_data=False, one_hot=False):
|
||||
class DataSets(object):
|
||||
pass
|
||||
data_sets = DataSets()
|
||||
if fake_data:
|
||||
data_sets.train = DataSet([], [], fake_data=True)
|
||||
data_sets.validation = DataSet([], [], fake_data=True)
|
||||
data_sets.test = DataSet([], [], fake_data=True)
|
||||
return data_sets
|
||||
TRAIN_IMAGES = 'train-images-idx3-ubyte.gz'
|
||||
TRAIN_LABELS = 'train-labels-idx1-ubyte.gz'
|
||||
TEST_IMAGES = 't10k-images-idx3-ubyte.gz'
|
||||
TEST_LABELS = 't10k-labels-idx1-ubyte.gz'
|
||||
VALIDATION_SIZE = 5000
|
||||
local_file = maybe_download(TRAIN_IMAGES, train_dir)
|
||||
train_images = extract_images(local_file)
|
||||
local_file = maybe_download(TRAIN_LABELS, train_dir)
|
||||
train_labels = extract_labels(local_file, one_hot=one_hot)
|
||||
local_file = maybe_download(TEST_IMAGES, train_dir)
|
||||
test_images = extract_images(local_file)
|
||||
local_file = maybe_download(TEST_LABELS, train_dir)
|
||||
test_labels = extract_labels(local_file, one_hot=one_hot)
|
||||
validation_images = train_images[:VALIDATION_SIZE]
|
||||
validation_labels = train_labels[:VALIDATION_SIZE]
|
||||
train_images = train_images[VALIDATION_SIZE:]
|
||||
train_labels = train_labels[VALIDATION_SIZE:]
|
||||
data_sets.train = DataSet(train_images, train_labels)
|
||||
data_sets.validation = DataSet(validation_images, validation_labels)
|
||||
data_sets.test = DataSet(test_images, test_labels)
|
||||
return data_sets
|
||||
85
deep-learning/tensor-flow-examples/multigpu_basics.py
Normal file
85
deep-learning/tensor-flow-examples/multigpu_basics.py
Normal file
|
|
@ -0,0 +1,85 @@
|
|||
#Multi GPU Basic example
|
||||
'''
|
||||
This tutorial requires your machine to have 2 GPUs
|
||||
"/cpu:0": The CPU of your machine.
|
||||
"/gpu:0": The first GPU of your machine
|
||||
"/gpu:1": The second GPU of your machine
|
||||
'''
|
||||
|
||||
import numpy as np
|
||||
import tensorflow as tf
|
||||
import datetime
|
||||
|
||||
#Processing Units logs
|
||||
log_device_placement = True
|
||||
|
||||
#num of multiplications to perform
|
||||
n = 10
|
||||
|
||||
'''
|
||||
Example: compute A^n + B^n on 2 GPUs
|
||||
Results on 8 cores with 2 GTX-980:
|
||||
* Single GPU computation time: 0:00:11.277449
|
||||
* Multi GPU computation time: 0:00:07.131701
|
||||
'''
|
||||
#Create random large matrix
|
||||
A = np.random.rand(1e4, 1e4).astype('float32')
|
||||
B = np.random.rand(1e4, 1e4).astype('float32')
|
||||
|
||||
# Creates a graph to store results
|
||||
c1 = []
|
||||
c2 = []
|
||||
|
||||
def matpow(M, n):
|
||||
if n < 1: #Abstract cases where n < 1
|
||||
return M
|
||||
else:
|
||||
return tf.matmul(M, matpow(M, n-1))
|
||||
|
||||
'''
|
||||
Single GPU computing
|
||||
'''
|
||||
with tf.device('/gpu:0'):
|
||||
a = tf.constant(A)
|
||||
b = tf.constant(B)
|
||||
#compute A^n and B^n and store results in c1
|
||||
c1.append(matpow(a, n))
|
||||
c1.append(matpow(b, n))
|
||||
|
||||
with tf.device('/cpu:0'):
|
||||
sum = tf.add_n(c1) #Addition of all elements in c1, i.e. A^n + B^n
|
||||
|
||||
t1_1 = datetime.datetime.now()
|
||||
with tf.Session(config=tf.ConfigProto(log_device_placement=log_device_placement)) as sess:
|
||||
# Runs the op.
|
||||
sess.run(sum)
|
||||
t2_1 = datetime.datetime.now()
|
||||
|
||||
|
||||
'''
|
||||
Multi GPU computing
|
||||
'''
|
||||
#GPU:0 computes A^n
|
||||
with tf.device('/gpu:0'):
|
||||
#compute A^n and store result in c2
|
||||
a = tf.constant(A)
|
||||
c2.append(matpow(a, n))
|
||||
|
||||
#GPU:1 computes B^n
|
||||
with tf.device('/gpu:1'):
|
||||
#compute B^n and store result in c2
|
||||
b = tf.constant(B)
|
||||
c2.append(matpow(b, n))
|
||||
|
||||
with tf.device('/cpu:0'):
|
||||
sum = tf.add_n(c2) #Addition of all elements in c2, i.e. A^n + B^n
|
||||
|
||||
t1_2 = datetime.datetime.now()
|
||||
with tf.Session(config=tf.ConfigProto(log_device_placement=log_device_placement)) as sess:
|
||||
# Runs the op.
|
||||
sess.run(sum)
|
||||
t2_2 = datetime.datetime.now()
|
||||
|
||||
|
||||
print "Single GPU computation time: " + str(t2_1-t1_1)
|
||||
print "Multi GPU computation time: " + str(t2_2-t1_2)
|
||||
|
|
@ -0,0 +1,220 @@
|
|||
{
|
||||
"cells": [
|
||||
{
|
||||
"cell_type": "markdown",
|
||||
"metadata": {},
|
||||
"source": [
|
||||
"# Basic Operations in TensorFlow\n",
|
||||
"\n",
|
||||
"Credits: Forked from [TensorFlow-Examples](https://github.com/aymericdamien/TensorFlow-Examples) by Aymeric Damien\n",
|
||||
"\n",
|
||||
"## Setup\n",
|
||||
"\n",
|
||||
"Refer to the [setup instructions](http://nbviewer.ipython.org/github/donnemartin/data-science-ipython-notebooks/blob/master/deep-learning/tensor-flow-examples/Setup_TensorFlow.md)"
|
||||
]
|
||||
},
|
||||
{
|
||||
"cell_type": "code",
|
||||
"execution_count": 2,
|
||||
"metadata": {
|
||||
"collapsed": true
|
||||
},
|
||||
"outputs": [],
|
||||
"source": [
|
||||
"import tensorflow as tf"
|
||||
]
|
||||
},
|
||||
{
|
||||
"cell_type": "code",
|
||||
"execution_count": 3,
|
||||
"metadata": {
|
||||
"collapsed": true
|
||||
},
|
||||
"outputs": [],
|
||||
"source": [
|
||||
"# Basic constant operations\n",
|
||||
"# The value returned by the constructor represents the output\n",
|
||||
"# of the Constant op.\n",
|
||||
"a = tf.constant(2)\n",
|
||||
"b = tf.constant(3)"
|
||||
]
|
||||
},
|
||||
{
|
||||
"cell_type": "code",
|
||||
"execution_count": 4,
|
||||
"metadata": {
|
||||
"collapsed": false
|
||||
},
|
||||
"outputs": [
|
||||
{
|
||||
"name": "stdout",
|
||||
"output_type": "stream",
|
||||
"text": [
|
||||
"a=2, b=3\n",
|
||||
"Addition with constants: 5\n",
|
||||
"Multiplication with constants: 6\n"
|
||||
]
|
||||
}
|
||||
],
|
||||
"source": [
|
||||
"# Launch the default graph.\n",
|
||||
"with tf.Session() as sess:\n",
|
||||
" print \"a=2, b=3\"\n",
|
||||
" print \"Addition with constants: %i\" % sess.run(a+b)\n",
|
||||
" print \"Multiplication with constants: %i\" % sess.run(a*b)"
|
||||
]
|
||||
},
|
||||
{
|
||||
"cell_type": "code",
|
||||
"execution_count": 5,
|
||||
"metadata": {
|
||||
"collapsed": true
|
||||
},
|
||||
"outputs": [],
|
||||
"source": [
|
||||
"# Basic Operations with variable as graph input\n",
|
||||
"# The value returned by the constructor represents the output\n",
|
||||
"# of the Variable op. (define as input when running session)\n",
|
||||
"# tf Graph input\n",
|
||||
"a = tf.placeholder(tf.types.int16)\n",
|
||||
"b = tf.placeholder(tf.types.int16)"
|
||||
]
|
||||
},
|
||||
{
|
||||
"cell_type": "code",
|
||||
"execution_count": 6,
|
||||
"metadata": {
|
||||
"collapsed": true
|
||||
},
|
||||
"outputs": [],
|
||||
"source": [
|
||||
"# Define some operations\n",
|
||||
"add = tf.add(a, b)\n",
|
||||
"mul = tf.mul(a, b)"
|
||||
]
|
||||
},
|
||||
{
|
||||
"cell_type": "code",
|
||||
"execution_count": 7,
|
||||
"metadata": {
|
||||
"collapsed": false
|
||||
},
|
||||
"outputs": [
|
||||
{
|
||||
"name": "stdout",
|
||||
"output_type": "stream",
|
||||
"text": [
|
||||
"Addition with variables: 5\n",
|
||||
"Multiplication with variables: 6\n"
|
||||
]
|
||||
}
|
||||
],
|
||||
"source": [
|
||||
"# Launch the default graph.\n",
|
||||
"with tf.Session() as sess:\n",
|
||||
" # Run every operation with variable input\n",
|
||||
" print \"Addition with variables: %i\" % sess.run(add, feed_dict={a: 2, b: 3})\n",
|
||||
" print \"Multiplication with variables: %i\" % sess.run(mul, feed_dict={a: 2, b: 3})"
|
||||
]
|
||||
},
|
||||
{
|
||||
"cell_type": "code",
|
||||
"execution_count": 8,
|
||||
"metadata": {
|
||||
"collapsed": true
|
||||
},
|
||||
"outputs": [],
|
||||
"source": [
|
||||
"# ----------------\n",
|
||||
"# More in details:\n",
|
||||
"# Matrix Multiplication from TensorFlow official tutorial\n",
|
||||
"\n",
|
||||
"# Create a Constant op that produces a 1x2 matrix. The op is\n",
|
||||
"# added as a node to the default graph.\n",
|
||||
"#\n",
|
||||
"# The value returned by the constructor represents the output\n",
|
||||
"# of the Constant op.\n",
|
||||
"matrix1 = tf.constant([[3., 3.]])"
|
||||
]
|
||||
},
|
||||
{
|
||||
"cell_type": "code",
|
||||
"execution_count": 9,
|
||||
"metadata": {
|
||||
"collapsed": true
|
||||
},
|
||||
"outputs": [],
|
||||
"source": [
|
||||
"# Create another Constant that produces a 2x1 matrix.\n",
|
||||
"matrix2 = tf.constant([[2.],[2.]])"
|
||||
]
|
||||
},
|
||||
{
|
||||
"cell_type": "code",
|
||||
"execution_count": 10,
|
||||
"metadata": {
|
||||
"collapsed": true
|
||||
},
|
||||
"outputs": [],
|
||||
"source": [
|
||||
"# Create a Matmul op that takes 'matrix1' and 'matrix2' as inputs.\n",
|
||||
"# The returned value, 'product', represents the result of the matrix\n",
|
||||
"# multiplication.\n",
|
||||
"product = tf.matmul(matrix1, matrix2)"
|
||||
]
|
||||
},
|
||||
{
|
||||
"cell_type": "code",
|
||||
"execution_count": 11,
|
||||
"metadata": {
|
||||
"collapsed": false
|
||||
},
|
||||
"outputs": [
|
||||
{
|
||||
"name": "stdout",
|
||||
"output_type": "stream",
|
||||
"text": [
|
||||
"[[ 12.]]\n"
|
||||
]
|
||||
}
|
||||
],
|
||||
"source": [
|
||||
"# To run the matmul op we call the session 'run()' method, passing 'product'\n",
|
||||
"# which represents the output of the matmul op. This indicates to the call\n",
|
||||
"# that we want to get the output of the matmul op back.\n",
|
||||
"#\n",
|
||||
"# All inputs needed by the op are run automatically by the session. They\n",
|
||||
"# typically are run in parallel.\n",
|
||||
"#\n",
|
||||
"# The call 'run(product)' thus causes the execution of threes ops in the\n",
|
||||
"# graph: the two constants and matmul.\n",
|
||||
"#\n",
|
||||
"# The output of the op is returned in 'result' as a numpy `ndarray` object.\n",
|
||||
"with tf.Session() as sess:\n",
|
||||
" result = sess.run(product)\n",
|
||||
" print result"
|
||||
]
|
||||
}
|
||||
],
|
||||
"metadata": {
|
||||
"kernelspec": {
|
||||
"display_name": "Python 3",
|
||||
"language": "python",
|
||||
"name": "python3"
|
||||
},
|
||||
"language_info": {
|
||||
"codemirror_mode": {
|
||||
"name": "ipython",
|
||||
"version": 3
|
||||
},
|
||||
"file_extension": ".py",
|
||||
"mimetype": "text/x-python",
|
||||
"name": "python",
|
||||
"nbconvert_exporter": "python",
|
||||
"pygments_lexer": "ipython3",
|
||||
"version": "3.4.3"
|
||||
}
|
||||
},
|
||||
"nbformat": 4,
|
||||
"nbformat_minor": 0
|
||||
}
|
||||
File diff suppressed because one or more lines are too long
|
|
@ -0,0 +1,227 @@
|
|||
{
|
||||
"cells": [
|
||||
{
|
||||
"cell_type": "markdown",
|
||||
"metadata": {
|
||||
"collapsed": true
|
||||
},
|
||||
"source": [
|
||||
"# Logistic Regression in TensorFlow\n",
|
||||
"\n",
|
||||
"Credits: Forked from [TensorFlow-Examples](https://github.com/aymericdamien/TensorFlow-Examples) by Aymeric Damien\n",
|
||||
"\n",
|
||||
"## Setup\n",
|
||||
"\n",
|
||||
"Refer to the [setup instructions](http://nbviewer.ipython.org/github/donnemartin/data-science-ipython-notebooks/blob/master/deep-learning/tensor-flow-examples/Setup_TensorFlow.md)"
|
||||
]
|
||||
},
|
||||
{
|
||||
"cell_type": "code",
|
||||
"execution_count": 5,
|
||||
"metadata": {
|
||||
"collapsed": false
|
||||
},
|
||||
"outputs": [
|
||||
{
|
||||
"name": "stdout",
|
||||
"output_type": "stream",
|
||||
"text": [
|
||||
"Extracting /tmp/data/train-images-idx3-ubyte.gz\n",
|
||||
"Extracting /tmp/data/train-labels-idx1-ubyte.gz\n",
|
||||
"Extracting /tmp/data/t10k-images-idx3-ubyte.gz\n",
|
||||
"Extracting /tmp/data/t10k-labels-idx1-ubyte.gz\n"
|
||||
]
|
||||
}
|
||||
],
|
||||
"source": [
|
||||
"# Import MINST data\n",
|
||||
"import input_data\n",
|
||||
"mnist = input_data.read_data_sets(\"/tmp/data/\", one_hot=True)"
|
||||
]
|
||||
},
|
||||
{
|
||||
"cell_type": "code",
|
||||
"execution_count": 6,
|
||||
"metadata": {
|
||||
"collapsed": true
|
||||
},
|
||||
"outputs": [],
|
||||
"source": [
|
||||
"import tensorflow as tf"
|
||||
]
|
||||
},
|
||||
{
|
||||
"cell_type": "code",
|
||||
"execution_count": 7,
|
||||
"metadata": {
|
||||
"collapsed": true
|
||||
},
|
||||
"outputs": [],
|
||||
"source": [
|
||||
"# Parameters\n",
|
||||
"learning_rate = 0.01\n",
|
||||
"training_epochs = 25\n",
|
||||
"batch_size = 100\n",
|
||||
"display_step = 1"
|
||||
]
|
||||
},
|
||||
{
|
||||
"cell_type": "code",
|
||||
"execution_count": 8,
|
||||
"metadata": {
|
||||
"collapsed": true
|
||||
},
|
||||
"outputs": [],
|
||||
"source": [
|
||||
"# tf Graph Input\n",
|
||||
"x = tf.placeholder(\"float\", [None, 784]) # mnist data image of shape 28*28=784\n",
|
||||
"y = tf.placeholder(\"float\", [None, 10]) # 0-9 digits recognition => 10 classes"
|
||||
]
|
||||
},
|
||||
{
|
||||
"cell_type": "code",
|
||||
"execution_count": 9,
|
||||
"metadata": {
|
||||
"collapsed": true
|
||||
},
|
||||
"outputs": [],
|
||||
"source": [
|
||||
"# Create model\n",
|
||||
"\n",
|
||||
"# Set model weights\n",
|
||||
"W = tf.Variable(tf.zeros([784, 10]))\n",
|
||||
"b = tf.Variable(tf.zeros([10]))"
|
||||
]
|
||||
},
|
||||
{
|
||||
"cell_type": "code",
|
||||
"execution_count": 10,
|
||||
"metadata": {
|
||||
"collapsed": true
|
||||
},
|
||||
"outputs": [],
|
||||
"source": [
|
||||
"# Construct model\n",
|
||||
"activation = tf.nn.softmax(tf.matmul(x, W) + b) # Softmax"
|
||||
]
|
||||
},
|
||||
{
|
||||
"cell_type": "code",
|
||||
"execution_count": 11,
|
||||
"metadata": {
|
||||
"collapsed": true
|
||||
},
|
||||
"outputs": [],
|
||||
"source": [
|
||||
"# Minimize error using cross entropy\n",
|
||||
"# Cross entropy\n",
|
||||
"cost = -tf.reduce_sum(y*tf.log(activation)) \n",
|
||||
"# Gradient Descent\n",
|
||||
"optimizer = tf.train.GradientDescentOptimizer(learning_rate).minimize(cost) "
|
||||
]
|
||||
},
|
||||
{
|
||||
"cell_type": "code",
|
||||
"execution_count": 12,
|
||||
"metadata": {
|
||||
"collapsed": true
|
||||
},
|
||||
"outputs": [],
|
||||
"source": [
|
||||
"# Initializing the variables\n",
|
||||
"init = tf.initialize_all_variables()"
|
||||
]
|
||||
},
|
||||
{
|
||||
"cell_type": "code",
|
||||
"execution_count": 13,
|
||||
"metadata": {
|
||||
"collapsed": false
|
||||
},
|
||||
"outputs": [
|
||||
{
|
||||
"name": "stdout",
|
||||
"output_type": "stream",
|
||||
"text": [
|
||||
"Epoch: 0001 cost= 29.860479714\n",
|
||||
"Epoch: 0002 cost= 22.080549484\n",
|
||||
"Epoch: 0003 cost= 21.237104595\n",
|
||||
"Epoch: 0004 cost= 20.460196280\n",
|
||||
"Epoch: 0005 cost= 20.185128237\n",
|
||||
"Epoch: 0006 cost= 19.940297202\n",
|
||||
"Epoch: 0007 cost= 19.645111119\n",
|
||||
"Epoch: 0008 cost= 19.507218031\n",
|
||||
"Epoch: 0009 cost= 19.389794492\n",
|
||||
"Epoch: 0010 cost= 19.177005816\n",
|
||||
"Epoch: 0011 cost= 19.082493615\n",
|
||||
"Epoch: 0012 cost= 19.072873598\n",
|
||||
"Epoch: 0013 cost= 18.938005402\n",
|
||||
"Epoch: 0014 cost= 18.891806430\n",
|
||||
"Epoch: 0015 cost= 18.839480221\n",
|
||||
"Epoch: 0016 cost= 18.769349510\n",
|
||||
"Epoch: 0017 cost= 18.590865587\n",
|
||||
"Epoch: 0018 cost= 18.623413677\n",
|
||||
"Epoch: 0019 cost= 18.546149085\n",
|
||||
"Epoch: 0020 cost= 18.432274895\n",
|
||||
"Epoch: 0021 cost= 18.358189004\n",
|
||||
"Epoch: 0022 cost= 18.380014628\n",
|
||||
"Epoch: 0023 cost= 18.499993471\n",
|
||||
"Epoch: 0024 cost= 18.386477311\n",
|
||||
"Epoch: 0025 cost= 18.258080609\n",
|
||||
"Optimization Finished!\n",
|
||||
"Accuracy: 0.9048\n"
|
||||
]
|
||||
}
|
||||
],
|
||||
"source": [
|
||||
"# Launch the graph\n",
|
||||
"with tf.Session() as sess:\n",
|
||||
" sess.run(init)\n",
|
||||
"\n",
|
||||
" # Training cycle\n",
|
||||
" for epoch in range(training_epochs):\n",
|
||||
" avg_cost = 0.\n",
|
||||
" total_batch = int(mnist.train.num_examples/batch_size)\n",
|
||||
" # Loop over all batches\n",
|
||||
" for i in range(total_batch):\n",
|
||||
" batch_xs, batch_ys = mnist.train.next_batch(batch_size)\n",
|
||||
" # Fit training using batch data\n",
|
||||
" sess.run(optimizer, feed_dict={x: batch_xs, y: batch_ys})\n",
|
||||
" # Compute average loss\n",
|
||||
" avg_cost += sess.run(cost, feed_dict={x: batch_xs, y: batch_ys})/total_batch\n",
|
||||
" # Display logs per epoch step\n",
|
||||
" if epoch % display_step == 0:\n",
|
||||
" print \"Epoch:\", '%04d' % (epoch+1), \"cost=\", \"{:.9f}\".format(avg_cost)\n",
|
||||
"\n",
|
||||
" print \"Optimization Finished!\"\n",
|
||||
"\n",
|
||||
" # Test model\n",
|
||||
" correct_prediction = tf.equal(tf.argmax(activation, 1), tf.argmax(y, 1))\n",
|
||||
" # Calculate accuracy\n",
|
||||
" accuracy = tf.reduce_mean(tf.cast(correct_prediction, \"float\"))\n",
|
||||
" print \"Accuracy:\", accuracy.eval({x: mnist.test.images, y: mnist.test.labels})"
|
||||
]
|
||||
}
|
||||
],
|
||||
"metadata": {
|
||||
"kernelspec": {
|
||||
"display_name": "Python 3",
|
||||
"language": "python",
|
||||
"name": "python3"
|
||||
},
|
||||
"language_info": {
|
||||
"codemirror_mode": {
|
||||
"name": "ipython",
|
||||
"version": 3
|
||||
},
|
||||
"file_extension": ".py",
|
||||
"mimetype": "text/x-python",
|
||||
"name": "python",
|
||||
"nbconvert_exporter": "python",
|
||||
"pygments_lexer": "ipython3",
|
||||
"version": "3.4.3"
|
||||
}
|
||||
},
|
||||
"nbformat": 4,
|
||||
"nbformat_minor": 0
|
||||
}
|
||||
|
|
@ -0,0 +1,386 @@
|
|||
{
|
||||
"cells": [
|
||||
{
|
||||
"cell_type": "markdown",
|
||||
"metadata": {},
|
||||
"source": [
|
||||
"# Nearest Neighbor in TensorFlow\n",
|
||||
"\n",
|
||||
"Credits: Forked from [TensorFlow-Examples](https://github.com/aymericdamien/TensorFlow-Examples) by Aymeric Damien\n",
|
||||
"\n",
|
||||
"## Setup\n",
|
||||
"\n",
|
||||
"Refer to the [setup instructions](http://nbviewer.ipython.org/github/donnemartin/data-science-ipython-notebooks/blob/master/deep-learning/tensor-flow-examples/Setup_TensorFlow.md)"
|
||||
]
|
||||
},
|
||||
{
|
||||
"cell_type": "code",
|
||||
"execution_count": 2,
|
||||
"metadata": {
|
||||
"collapsed": true
|
||||
},
|
||||
"outputs": [],
|
||||
"source": [
|
||||
"import numpy as np\n",
|
||||
"import tensorflow as tf"
|
||||
]
|
||||
},
|
||||
{
|
||||
"cell_type": "code",
|
||||
"execution_count": 3,
|
||||
"metadata": {
|
||||
"collapsed": false
|
||||
},
|
||||
"outputs": [
|
||||
{
|
||||
"name": "stdout",
|
||||
"output_type": "stream",
|
||||
"text": [
|
||||
"Extracting /tmp/data/train-images-idx3-ubyte.gz\n",
|
||||
"Extracting /tmp/data/train-labels-idx1-ubyte.gz\n",
|
||||
"Extracting /tmp/data/t10k-images-idx3-ubyte.gz\n",
|
||||
"Extracting /tmp/data/t10k-labels-idx1-ubyte.gz\n"
|
||||
]
|
||||
}
|
||||
],
|
||||
"source": [
|
||||
"# Import MINST data\n",
|
||||
"import input_data\n",
|
||||
"mnist = input_data.read_data_sets(\"/tmp/data/\", one_hot=True)"
|
||||
]
|
||||
},
|
||||
{
|
||||
"cell_type": "code",
|
||||
"execution_count": 4,
|
||||
"metadata": {
|
||||
"collapsed": true
|
||||
},
|
||||
"outputs": [],
|
||||
"source": [
|
||||
"# In this example, we limit mnist data\n",
|
||||
"Xtr, Ytr = mnist.train.next_batch(5000) #5000 for training (nn candidates)\n",
|
||||
"Xte, Yte = mnist.test.next_batch(200) #200 for testing"
|
||||
]
|
||||
},
|
||||
{
|
||||
"cell_type": "code",
|
||||
"execution_count": 5,
|
||||
"metadata": {
|
||||
"collapsed": true
|
||||
},
|
||||
"outputs": [],
|
||||
"source": [
|
||||
"# Reshape images to 1D\n",
|
||||
"Xtr = np.reshape(Xtr, newshape=(-1, 28*28))\n",
|
||||
"Xte = np.reshape(Xte, newshape=(-1, 28*28))"
|
||||
]
|
||||
},
|
||||
{
|
||||
"cell_type": "code",
|
||||
"execution_count": 6,
|
||||
"metadata": {
|
||||
"collapsed": true
|
||||
},
|
||||
"outputs": [],
|
||||
"source": [
|
||||
"# tf Graph Input\n",
|
||||
"xtr = tf.placeholder(\"float\", [None, 784])\n",
|
||||
"xte = tf.placeholder(\"float\", [784])"
|
||||
]
|
||||
},
|
||||
{
|
||||
"cell_type": "code",
|
||||
"execution_count": 8,
|
||||
"metadata": {
|
||||
"collapsed": true
|
||||
},
|
||||
"outputs": [],
|
||||
"source": [
|
||||
"# Nearest Neighbor calculation using L1 Distance\n",
|
||||
"# Calculate L1 Distance\n",
|
||||
"distance = tf.reduce_sum(tf.abs(tf.add(xtr, tf.neg(xte))), reduction_indices=1)\n",
|
||||
"# Predict: Get min distance index (Nearest neighbor)\n",
|
||||
"pred = tf.arg_min(distance, 0)\n",
|
||||
"\n",
|
||||
"accuracy = 0."
|
||||
]
|
||||
},
|
||||
{
|
||||
"cell_type": "code",
|
||||
"execution_count": 9,
|
||||
"metadata": {
|
||||
"collapsed": true
|
||||
},
|
||||
"outputs": [],
|
||||
"source": [
|
||||
"# Initializing the variables\n",
|
||||
"init = tf.initialize_all_variables()"
|
||||
]
|
||||
},
|
||||
{
|
||||
"cell_type": "code",
|
||||
"execution_count": 10,
|
||||
"metadata": {
|
||||
"collapsed": false
|
||||
},
|
||||
"outputs": [
|
||||
{
|
||||
"name": "stdout",
|
||||
"output_type": "stream",
|
||||
"text": [
|
||||
"Test 0 Prediction: 7 True Class: 7\n",
|
||||
"Test 1 Prediction: 2 True Class: 2\n",
|
||||
"Test 2 Prediction: 1 True Class: 1\n",
|
||||
"Test 3 Prediction: 0 True Class: 0\n",
|
||||
"Test 4 Prediction: 4 True Class: 4\n",
|
||||
"Test 5 Prediction: 1 True Class: 1\n",
|
||||
"Test 6 Prediction: 4 True Class: 4\n",
|
||||
"Test 7 Prediction: 9 True Class: 9\n",
|
||||
"Test 8 Prediction: 8 True Class: 5\n",
|
||||
"Test 9 Prediction: 9 True Class: 9\n",
|
||||
"Test 10 Prediction: 0 True Class: 0\n",
|
||||
"Test 11 Prediction: 0 True Class: 6\n",
|
||||
"Test 12 Prediction: 9 True Class: 9\n",
|
||||
"Test 13 Prediction: 0 True Class: 0\n",
|
||||
"Test 14 Prediction: 1 True Class: 1\n",
|
||||
"Test 15 Prediction: 5 True Class: 5\n",
|
||||
"Test 16 Prediction: 4 True Class: 9\n",
|
||||
"Test 17 Prediction: 7 True Class: 7\n",
|
||||
"Test 18 Prediction: 3 True Class: 3\n",
|
||||
"Test 19 Prediction: 4 True Class: 4\n",
|
||||
"Test 20 Prediction: 9 True Class: 9\n",
|
||||
"Test 21 Prediction: 6 True Class: 6\n",
|
||||
"Test 22 Prediction: 6 True Class: 6\n",
|
||||
"Test 23 Prediction: 5 True Class: 5\n",
|
||||
"Test 24 Prediction: 4 True Class: 4\n",
|
||||
"Test 25 Prediction: 0 True Class: 0\n",
|
||||
"Test 26 Prediction: 7 True Class: 7\n",
|
||||
"Test 27 Prediction: 4 True Class: 4\n",
|
||||
"Test 28 Prediction: 0 True Class: 0\n",
|
||||
"Test 29 Prediction: 1 True Class: 1\n",
|
||||
"Test 30 Prediction: 3 True Class: 3\n",
|
||||
"Test 31 Prediction: 1 True Class: 1\n",
|
||||
"Test 32 Prediction: 3 True Class: 3\n",
|
||||
"Test 33 Prediction: 4 True Class: 4\n",
|
||||
"Test 34 Prediction: 7 True Class: 7\n",
|
||||
"Test 35 Prediction: 2 True Class: 2\n",
|
||||
"Test 36 Prediction: 7 True Class: 7\n",
|
||||
"Test 37 Prediction: 1 True Class: 1\n",
|
||||
"Test 38 Prediction: 2 True Class: 2\n",
|
||||
"Test 39 Prediction: 1 True Class: 1\n",
|
||||
"Test 40 Prediction: 1 True Class: 1\n",
|
||||
"Test 41 Prediction: 7 True Class: 7\n",
|
||||
"Test 42 Prediction: 4 True Class: 4\n",
|
||||
"Test 43 Prediction: 1 True Class: 2\n",
|
||||
"Test 44 Prediction: 3 True Class: 3\n",
|
||||
"Test 45 Prediction: 5 True Class: 5\n",
|
||||
"Test 46 Prediction: 1 True Class: 1\n",
|
||||
"Test 47 Prediction: 2 True Class: 2\n",
|
||||
"Test 48 Prediction: 4 True Class: 4\n",
|
||||
"Test 49 Prediction: 4 True Class: 4\n",
|
||||
"Test 50 Prediction: 6 True Class: 6\n",
|
||||
"Test 51 Prediction: 3 True Class: 3\n",
|
||||
"Test 52 Prediction: 5 True Class: 5\n",
|
||||
"Test 53 Prediction: 5 True Class: 5\n",
|
||||
"Test 54 Prediction: 6 True Class: 6\n",
|
||||
"Test 55 Prediction: 0 True Class: 0\n",
|
||||
"Test 56 Prediction: 4 True Class: 4\n",
|
||||
"Test 57 Prediction: 1 True Class: 1\n",
|
||||
"Test 58 Prediction: 9 True Class: 9\n",
|
||||
"Test 59 Prediction: 5 True Class: 5\n",
|
||||
"Test 60 Prediction: 7 True Class: 7\n",
|
||||
"Test 61 Prediction: 8 True Class: 8\n",
|
||||
"Test 62 Prediction: 9 True Class: 9\n",
|
||||
"Test 63 Prediction: 3 True Class: 3\n",
|
||||
"Test 64 Prediction: 7 True Class: 7\n",
|
||||
"Test 65 Prediction: 4 True Class: 4\n",
|
||||
"Test 66 Prediction: 6 True Class: 6\n",
|
||||
"Test 67 Prediction: 4 True Class: 4\n",
|
||||
"Test 68 Prediction: 3 True Class: 3\n",
|
||||
"Test 69 Prediction: 0 True Class: 0\n",
|
||||
"Test 70 Prediction: 7 True Class: 7\n",
|
||||
"Test 71 Prediction: 0 True Class: 0\n",
|
||||
"Test 72 Prediction: 2 True Class: 2\n",
|
||||
"Test 73 Prediction: 7 True Class: 9\n",
|
||||
"Test 74 Prediction: 1 True Class: 1\n",
|
||||
"Test 75 Prediction: 7 True Class: 7\n",
|
||||
"Test 76 Prediction: 3 True Class: 3\n",
|
||||
"Test 77 Prediction: 7 True Class: 2\n",
|
||||
"Test 78 Prediction: 9 True Class: 9\n",
|
||||
"Test 79 Prediction: 7 True Class: 7\n",
|
||||
"Test 80 Prediction: 7 True Class: 7\n",
|
||||
"Test 81 Prediction: 6 True Class: 6\n",
|
||||
"Test 82 Prediction: 2 True Class: 2\n",
|
||||
"Test 83 Prediction: 7 True Class: 7\n",
|
||||
"Test 84 Prediction: 8 True Class: 8\n",
|
||||
"Test 85 Prediction: 4 True Class: 4\n",
|
||||
"Test 86 Prediction: 7 True Class: 7\n",
|
||||
"Test 87 Prediction: 3 True Class: 3\n",
|
||||
"Test 88 Prediction: 6 True Class: 6\n",
|
||||
"Test 89 Prediction: 1 True Class: 1\n",
|
||||
"Test 90 Prediction: 3 True Class: 3\n",
|
||||
"Test 91 Prediction: 6 True Class: 6\n",
|
||||
"Test 92 Prediction: 9 True Class: 9\n",
|
||||
"Test 93 Prediction: 3 True Class: 3\n",
|
||||
"Test 94 Prediction: 1 True Class: 1\n",
|
||||
"Test 95 Prediction: 4 True Class: 4\n",
|
||||
"Test 96 Prediction: 1 True Class: 1\n",
|
||||
"Test 97 Prediction: 7 True Class: 7\n",
|
||||
"Test 98 Prediction: 6 True Class: 6\n",
|
||||
"Test 99 Prediction: 9 True Class: 9\n",
|
||||
"Test 100 Prediction: 6 True Class: 6\n",
|
||||
"Test 101 Prediction: 0 True Class: 0\n",
|
||||
"Test 102 Prediction: 5 True Class: 5\n",
|
||||
"Test 103 Prediction: 4 True Class: 4\n",
|
||||
"Test 104 Prediction: 9 True Class: 9\n",
|
||||
"Test 105 Prediction: 9 True Class: 9\n",
|
||||
"Test 106 Prediction: 2 True Class: 2\n",
|
||||
"Test 107 Prediction: 1 True Class: 1\n",
|
||||
"Test 108 Prediction: 9 True Class: 9\n",
|
||||
"Test 109 Prediction: 4 True Class: 4\n",
|
||||
"Test 110 Prediction: 8 True Class: 8\n",
|
||||
"Test 111 Prediction: 7 True Class: 7\n",
|
||||
"Test 112 Prediction: 3 True Class: 3\n",
|
||||
"Test 113 Prediction: 9 True Class: 9\n",
|
||||
"Test 114 Prediction: 7 True Class: 7\n",
|
||||
"Test 115 Prediction: 9 True Class: 4\n",
|
||||
"Test 116 Prediction: 9 True Class: 4\n",
|
||||
"Test 117 Prediction: 4 True Class: 4\n",
|
||||
"Test 118 Prediction: 9 True Class: 9\n",
|
||||
"Test 119 Prediction: 7 True Class: 2\n",
|
||||
"Test 120 Prediction: 5 True Class: 5\n",
|
||||
"Test 121 Prediction: 4 True Class: 4\n",
|
||||
"Test 122 Prediction: 7 True Class: 7\n",
|
||||
"Test 123 Prediction: 6 True Class: 6\n",
|
||||
"Test 124 Prediction: 7 True Class: 7\n",
|
||||
"Test 125 Prediction: 9 True Class: 9\n",
|
||||
"Test 126 Prediction: 0 True Class: 0\n",
|
||||
"Test 127 Prediction: 5 True Class: 5\n",
|
||||
"Test 128 Prediction: 8 True Class: 8\n",
|
||||
"Test 129 Prediction: 5 True Class: 5\n",
|
||||
"Test 130 Prediction: 6 True Class: 6\n",
|
||||
"Test 131 Prediction: 6 True Class: 6\n",
|
||||
"Test 132 Prediction: 5 True Class: 5\n",
|
||||
"Test 133 Prediction: 7 True Class: 7\n",
|
||||
"Test 134 Prediction: 8 True Class: 8\n",
|
||||
"Test 135 Prediction: 1 True Class: 1\n",
|
||||
"Test 136 Prediction: 0 True Class: 0\n",
|
||||
"Test 137 Prediction: 1 True Class: 1\n",
|
||||
"Test 138 Prediction: 6 True Class: 6\n",
|
||||
"Test 139 Prediction: 4 True Class: 4\n",
|
||||
"Test 140 Prediction: 6 True Class: 6\n",
|
||||
"Test 141 Prediction: 7 True Class: 7\n",
|
||||
"Test 142 Prediction: 2 True Class: 3\n",
|
||||
"Test 143 Prediction: 1 True Class: 1\n",
|
||||
"Test 144 Prediction: 7 True Class: 7\n",
|
||||
"Test 145 Prediction: 1 True Class: 1\n",
|
||||
"Test 146 Prediction: 8 True Class: 8\n",
|
||||
"Test 147 Prediction: 2 True Class: 2\n",
|
||||
"Test 148 Prediction: 0 True Class: 0\n",
|
||||
"Test 149 Prediction: 1 True Class: 2\n",
|
||||
"Test 150 Prediction: 9 True Class: 9\n",
|
||||
"Test 151 Prediction: 9 True Class: 9\n",
|
||||
"Test 152 Prediction: 5 True Class: 5\n",
|
||||
"Test 153 Prediction: 5 True Class: 5\n",
|
||||
"Test 154 Prediction: 1 True Class: 1\n",
|
||||
"Test 155 Prediction: 5 True Class: 5\n",
|
||||
"Test 156 Prediction: 6 True Class: 6\n",
|
||||
"Test 157 Prediction: 0 True Class: 0\n",
|
||||
"Test 158 Prediction: 3 True Class: 3\n",
|
||||
"Test 159 Prediction: 4 True Class: 4\n",
|
||||
"Test 160 Prediction: 4 True Class: 4\n",
|
||||
"Test 161 Prediction: 6 True Class: 6\n",
|
||||
"Test 162 Prediction: 5 True Class: 5\n",
|
||||
"Test 163 Prediction: 4 True Class: 4\n",
|
||||
"Test 164 Prediction: 6 True Class: 6\n",
|
||||
"Test 165 Prediction: 5 True Class: 5\n",
|
||||
"Test 166 Prediction: 4 True Class: 4\n",
|
||||
"Test 167 Prediction: 5 True Class: 5\n",
|
||||
"Test 168 Prediction: 1 True Class: 1\n",
|
||||
"Test 169 Prediction: 4 True Class: 4\n",
|
||||
"Test 170 Prediction: 9 True Class: 4\n",
|
||||
"Test 171 Prediction: 7 True Class: 7\n",
|
||||
"Test 172 Prediction: 2 True Class: 2\n",
|
||||
"Test 173 Prediction: 3 True Class: 3\n",
|
||||
"Test 174 Prediction: 2 True Class: 2\n",
|
||||
"Test 175 Prediction: 1 True Class: 7\n",
|
||||
"Test 176 Prediction: 1 True Class: 1\n",
|
||||
"Test 177 Prediction: 8 True Class: 8\n",
|
||||
"Test 178 Prediction: 1 True Class: 1\n",
|
||||
"Test 179 Prediction: 8 True Class: 8\n",
|
||||
"Test 180 Prediction: 1 True Class: 1\n",
|
||||
"Test 181 Prediction: 8 True Class: 8\n",
|
||||
"Test 182 Prediction: 5 True Class: 5\n",
|
||||
"Test 183 Prediction: 0 True Class: 0\n",
|
||||
"Test 184 Prediction: 2 True Class: 8\n",
|
||||
"Test 185 Prediction: 9 True Class: 9\n",
|
||||
"Test 186 Prediction: 2 True Class: 2\n",
|
||||
"Test 187 Prediction: 5 True Class: 5\n",
|
||||
"Test 188 Prediction: 0 True Class: 0\n",
|
||||
"Test 189 Prediction: 1 True Class: 1\n",
|
||||
"Test 190 Prediction: 1 True Class: 1\n",
|
||||
"Test 191 Prediction: 1 True Class: 1\n",
|
||||
"Test 192 Prediction: 0 True Class: 0\n",
|
||||
"Test 193 Prediction: 4 True Class: 9\n",
|
||||
"Test 194 Prediction: 0 True Class: 0\n",
|
||||
"Test 195 Prediction: 1 True Class: 3\n",
|
||||
"Test 196 Prediction: 1 True Class: 1\n",
|
||||
"Test 197 Prediction: 6 True Class: 6\n",
|
||||
"Test 198 Prediction: 4 True Class: 4\n",
|
||||
"Test 199 Prediction: 2 True Class: 2\n",
|
||||
"Done!\n",
|
||||
"Accuracy: 0.92\n"
|
||||
]
|
||||
}
|
||||
],
|
||||
"source": [
|
||||
"# Launch the graph\n",
|
||||
"with tf.Session() as sess:\n",
|
||||
" sess.run(init)\n",
|
||||
"\n",
|
||||
" # loop over test data\n",
|
||||
" for i in range(len(Xte)):\n",
|
||||
" # Get nearest neighbor\n",
|
||||
" nn_index = sess.run(pred, feed_dict={xtr: Xtr, xte: Xte[i,:]})\n",
|
||||
" # Get nearest neighbor class label and compare it to its true label\n",
|
||||
" print \"Test\", i, \"Prediction:\", np.argmax(Ytr[nn_index]), \\\n",
|
||||
" \"True Class:\", np.argmax(Yte[i])\n",
|
||||
" # Calculate accuracy\n",
|
||||
" if np.argmax(Ytr[nn_index]) == np.argmax(Yte[i]):\n",
|
||||
" accuracy += 1./len(Xte)\n",
|
||||
" print \"Done!\"\n",
|
||||
" print \"Accuracy:\", accuracy"
|
||||
]
|
||||
},
|
||||
{
|
||||
"cell_type": "code",
|
||||
"execution_count": null,
|
||||
"metadata": {
|
||||
"collapsed": true
|
||||
},
|
||||
"outputs": [],
|
||||
"source": []
|
||||
}
|
||||
],
|
||||
"metadata": {
|
||||
"kernelspec": {
|
||||
"display_name": "Python 3",
|
||||
"language": "python",
|
||||
"name": "python3"
|
||||
},
|
||||
"language_info": {
|
||||
"codemirror_mode": {
|
||||
"name": "ipython",
|
||||
"version": 3
|
||||
},
|
||||
"file_extension": ".py",
|
||||
"mimetype": "text/x-python",
|
||||
"name": "python",
|
||||
"nbconvert_exporter": "python",
|
||||
"pygments_lexer": "ipython3",
|
||||
"version": "3.4.3"
|
||||
}
|
||||
},
|
||||
"nbformat": 4,
|
||||
"nbformat_minor": 0
|
||||
}
|
||||
|
|
@ -0,0 +1,345 @@
|
|||
{
|
||||
"cells": [
|
||||
{
|
||||
"cell_type": "markdown",
|
||||
"metadata": {},
|
||||
"source": [
|
||||
"# AlexNet in TensorFlow\n",
|
||||
"\n",
|
||||
"Credits: Forked from [TensorFlow-Examples](https://github.com/aymericdamien/TensorFlow-Examples) by Aymeric Damien\n",
|
||||
"\n",
|
||||
"## Setup\n",
|
||||
"\n",
|
||||
"Refer to the [setup instructions](http://nbviewer.ipython.org/github/donnemartin/data-science-ipython-notebooks/blob/master/deep-learning/tensor-flow-examples/Setup_TensorFlow.md)"
|
||||
]
|
||||
},
|
||||
{
|
||||
"cell_type": "code",
|
||||
"execution_count": 2,
|
||||
"metadata": {
|
||||
"collapsed": false
|
||||
},
|
||||
"outputs": [
|
||||
{
|
||||
"name": "stdout",
|
||||
"output_type": "stream",
|
||||
"text": [
|
||||
"Extracting /tmp/data/train-images-idx3-ubyte.gz\n",
|
||||
"Extracting /tmp/data/train-labels-idx1-ubyte.gz\n",
|
||||
"Extracting /tmp/data/t10k-images-idx3-ubyte.gz\n",
|
||||
"Extracting /tmp/data/t10k-labels-idx1-ubyte.gz\n"
|
||||
]
|
||||
}
|
||||
],
|
||||
"source": [
|
||||
"# Import MINST data\n",
|
||||
"import input_data\n",
|
||||
"mnist = input_data.read_data_sets(\"/tmp/data/\", one_hot=True)"
|
||||
]
|
||||
},
|
||||
{
|
||||
"cell_type": "code",
|
||||
"execution_count": 3,
|
||||
"metadata": {
|
||||
"collapsed": true
|
||||
},
|
||||
"outputs": [],
|
||||
"source": [
|
||||
"import tensorflow as tf"
|
||||
]
|
||||
},
|
||||
{
|
||||
"cell_type": "code",
|
||||
"execution_count": 17,
|
||||
"metadata": {
|
||||
"collapsed": true
|
||||
},
|
||||
"outputs": [],
|
||||
"source": [
|
||||
"# Parameters\n",
|
||||
"learning_rate = 0.001\n",
|
||||
"training_iters = 300000\n",
|
||||
"batch_size = 64\n",
|
||||
"display_step = 100"
|
||||
]
|
||||
},
|
||||
{
|
||||
"cell_type": "code",
|
||||
"execution_count": 5,
|
||||
"metadata": {
|
||||
"collapsed": true
|
||||
},
|
||||
"outputs": [],
|
||||
"source": [
|
||||
"# Network Parameters\n",
|
||||
"n_input = 784 # MNIST data input (img shape: 28*28)\n",
|
||||
"n_classes = 10 # MNIST total classes (0-9 digits)\n",
|
||||
"dropout = 0.8 # Dropout, probability to keep units"
|
||||
]
|
||||
},
|
||||
{
|
||||
"cell_type": "code",
|
||||
"execution_count": 6,
|
||||
"metadata": {
|
||||
"collapsed": true
|
||||
},
|
||||
"outputs": [],
|
||||
"source": [
|
||||
"# tf Graph input\n",
|
||||
"x = tf.placeholder(tf.types.float32, [None, n_input])\n",
|
||||
"y = tf.placeholder(tf.types.float32, [None, n_classes])\n",
|
||||
"keep_prob = tf.placeholder(tf.types.float32) # dropout (keep probability)"
|
||||
]
|
||||
},
|
||||
{
|
||||
"cell_type": "code",
|
||||
"execution_count": 7,
|
||||
"metadata": {
|
||||
"collapsed": true
|
||||
},
|
||||
"outputs": [],
|
||||
"source": [
|
||||
"# Create AlexNet model\n",
|
||||
"def conv2d(name, l_input, w, b):\n",
|
||||
" return tf.nn.relu(tf.nn.bias_add(tf.nn.conv2d(l_input, w, strides=[1, 1, 1, 1], \n",
|
||||
" padding='SAME'),b), name=name)\n",
|
||||
"\n",
|
||||
"def max_pool(name, l_input, k):\n",
|
||||
" return tf.nn.max_pool(l_input, ksize=[1, k, k, 1], strides=[1, k, k, 1], \n",
|
||||
" padding='SAME', name=name)\n",
|
||||
"\n",
|
||||
"def norm(name, l_input, lsize=4):\n",
|
||||
" return tf.nn.lrn(l_input, lsize, bias=1.0, alpha=0.001 / 9.0, beta=0.75, name=name)\n",
|
||||
"\n",
|
||||
"def alex_net(_X, _weights, _biases, _dropout):\n",
|
||||
" # Reshape input picture\n",
|
||||
" _X = tf.reshape(_X, shape=[-1, 28, 28, 1])\n",
|
||||
"\n",
|
||||
" # Convolution Layer\n",
|
||||
" conv1 = conv2d('conv1', _X, _weights['wc1'], _biases['bc1'])\n",
|
||||
" # Max Pooling (down-sampling)\n",
|
||||
" pool1 = max_pool('pool1', conv1, k=2)\n",
|
||||
" # Apply Normalization\n",
|
||||
" norm1 = norm('norm1', pool1, lsize=4)\n",
|
||||
" # Apply Dropout\n",
|
||||
" norm1 = tf.nn.dropout(norm1, _dropout)\n",
|
||||
"\n",
|
||||
" # Convolution Layer\n",
|
||||
" conv2 = conv2d('conv2', norm1, _weights['wc2'], _biases['bc2'])\n",
|
||||
" # Max Pooling (down-sampling)\n",
|
||||
" pool2 = max_pool('pool2', conv2, k=2)\n",
|
||||
" # Apply Normalization\n",
|
||||
" norm2 = norm('norm2', pool2, lsize=4)\n",
|
||||
" # Apply Dropout\n",
|
||||
" norm2 = tf.nn.dropout(norm2, _dropout)\n",
|
||||
"\n",
|
||||
" # Convolution Layer\n",
|
||||
" conv3 = conv2d('conv3', norm2, _weights['wc3'], _biases['bc3'])\n",
|
||||
" # Max Pooling (down-sampling)\n",
|
||||
" pool3 = max_pool('pool3', conv3, k=2)\n",
|
||||
" # Apply Normalization\n",
|
||||
" norm3 = norm('norm3', pool3, lsize=4)\n",
|
||||
" # Apply Dropout\n",
|
||||
" norm3 = tf.nn.dropout(norm3, _dropout)\n",
|
||||
"\n",
|
||||
" # Fully connected layer\n",
|
||||
" # Reshape conv3 output to fit dense layer input\n",
|
||||
" dense1 = tf.reshape(norm3, [-1, _weights['wd1'].get_shape().as_list()[0]]) \n",
|
||||
" # Relu activation\n",
|
||||
" dense1 = tf.nn.relu(tf.matmul(dense1, _weights['wd1']) + _biases['bd1'], name='fc1')\n",
|
||||
" \n",
|
||||
" # Relu activation\n",
|
||||
" dense2 = tf.nn.relu(tf.matmul(dense1, _weights['wd2']) + _biases['bd2'], name='fc2') \n",
|
||||
"\n",
|
||||
" # Output, class prediction\n",
|
||||
" out = tf.matmul(dense2, _weights['out']) + _biases['out']\n",
|
||||
" return out"
|
||||
]
|
||||
},
|
||||
{
|
||||
"cell_type": "code",
|
||||
"execution_count": 8,
|
||||
"metadata": {
|
||||
"collapsed": true
|
||||
},
|
||||
"outputs": [],
|
||||
"source": [
|
||||
"# Store layers weight & bias\n",
|
||||
"weights = {\n",
|
||||
" 'wc1': tf.Variable(tf.random_normal([3, 3, 1, 64])),\n",
|
||||
" 'wc2': tf.Variable(tf.random_normal([3, 3, 64, 128])),\n",
|
||||
" 'wc3': tf.Variable(tf.random_normal([3, 3, 128, 256])),\n",
|
||||
" 'wd1': tf.Variable(tf.random_normal([4*4*256, 1024])),\n",
|
||||
" 'wd2': tf.Variable(tf.random_normal([1024, 1024])),\n",
|
||||
" 'out': tf.Variable(tf.random_normal([1024, 10]))\n",
|
||||
"}\n",
|
||||
"biases = {\n",
|
||||
" 'bc1': tf.Variable(tf.random_normal([64])),\n",
|
||||
" 'bc2': tf.Variable(tf.random_normal([128])),\n",
|
||||
" 'bc3': tf.Variable(tf.random_normal([256])),\n",
|
||||
" 'bd1': tf.Variable(tf.random_normal([1024])),\n",
|
||||
" 'bd2': tf.Variable(tf.random_normal([1024])),\n",
|
||||
" 'out': tf.Variable(tf.random_normal([n_classes]))\n",
|
||||
"}"
|
||||
]
|
||||
},
|
||||
{
|
||||
"cell_type": "code",
|
||||
"execution_count": 9,
|
||||
"metadata": {
|
||||
"collapsed": true
|
||||
},
|
||||
"outputs": [],
|
||||
"source": [
|
||||
"# Construct model\n",
|
||||
"pred = alex_net(x, weights, biases, keep_prob)"
|
||||
]
|
||||
},
|
||||
{
|
||||
"cell_type": "code",
|
||||
"execution_count": 10,
|
||||
"metadata": {
|
||||
"collapsed": true
|
||||
},
|
||||
"outputs": [],
|
||||
"source": [
|
||||
"# Define loss and optimizer\n",
|
||||
"cost = tf.reduce_mean(tf.nn.softmax_cross_entropy_with_logits(pred, y))\n",
|
||||
"optimizer = tf.train.AdamOptimizer(learning_rate=learning_rate).minimize(cost)"
|
||||
]
|
||||
},
|
||||
{
|
||||
"cell_type": "code",
|
||||
"execution_count": 11,
|
||||
"metadata": {
|
||||
"collapsed": true
|
||||
},
|
||||
"outputs": [],
|
||||
"source": [
|
||||
"# Evaluate model\n",
|
||||
"correct_pred = tf.equal(tf.argmax(pred,1), tf.argmax(y,1))\n",
|
||||
"accuracy = tf.reduce_mean(tf.cast(correct_pred, tf.types.float32))"
|
||||
]
|
||||
},
|
||||
{
|
||||
"cell_type": "code",
|
||||
"execution_count": 12,
|
||||
"metadata": {
|
||||
"collapsed": true
|
||||
},
|
||||
"outputs": [],
|
||||
"source": [
|
||||
"# Initializing the variables\n",
|
||||
"init = tf.initialize_all_variables()"
|
||||
]
|
||||
},
|
||||
{
|
||||
"cell_type": "code",
|
||||
"execution_count": 18,
|
||||
"metadata": {
|
||||
"collapsed": false
|
||||
},
|
||||
"outputs": [
|
||||
{
|
||||
"name": "stdout",
|
||||
"output_type": "stream",
|
||||
"text": [
|
||||
"Iter 6400, Minibatch Loss= 29666.185547, Training Accuracy= 0.59375\n",
|
||||
"Iter 12800, Minibatch Loss= 22125.562500, Training Accuracy= 0.60938\n",
|
||||
"Iter 19200, Minibatch Loss= 22631.134766, Training Accuracy= 0.59375\n",
|
||||
"Iter 25600, Minibatch Loss= 18498.414062, Training Accuracy= 0.62500\n",
|
||||
"Iter 32000, Minibatch Loss= 11318.283203, Training Accuracy= 0.70312\n",
|
||||
"Iter 38400, Minibatch Loss= 12076.280273, Training Accuracy= 0.70312\n",
|
||||
"Iter 44800, Minibatch Loss= 8195.520508, Training Accuracy= 0.82812\n",
|
||||
"Iter 51200, Minibatch Loss= 5176.181641, Training Accuracy= 0.84375\n",
|
||||
"Iter 57600, Minibatch Loss= 8951.896484, Training Accuracy= 0.81250\n",
|
||||
"Iter 64000, Minibatch Loss= 10096.946289, Training Accuracy= 0.78125\n",
|
||||
"Iter 70400, Minibatch Loss= 11466.641602, Training Accuracy= 0.68750\n",
|
||||
"Iter 76800, Minibatch Loss= 7469.824219, Training Accuracy= 0.78125\n",
|
||||
"Iter 83200, Minibatch Loss= 4147.449219, Training Accuracy= 0.89062\n",
|
||||
"Iter 89600, Minibatch Loss= 5904.782227, Training Accuracy= 0.82812\n",
|
||||
"Iter 96000, Minibatch Loss= 718.493713, Training Accuracy= 0.93750\n",
|
||||
"Iter 102400, Minibatch Loss= 2184.151367, Training Accuracy= 0.93750\n",
|
||||
"Iter 108800, Minibatch Loss= 2354.463135, Training Accuracy= 0.89062\n",
|
||||
"Iter 115200, Minibatch Loss= 8612.959961, Training Accuracy= 0.81250\n",
|
||||
"Iter 121600, Minibatch Loss= 2225.773926, Training Accuracy= 0.84375\n",
|
||||
"Iter 128000, Minibatch Loss= 160.583618, Training Accuracy= 0.96875\n",
|
||||
"Iter 134400, Minibatch Loss= 1524.846069, Training Accuracy= 0.93750\n",
|
||||
"Iter 140800, Minibatch Loss= 3501.871094, Training Accuracy= 0.89062\n",
|
||||
"Iter 147200, Minibatch Loss= 661.977051, Training Accuracy= 0.96875\n",
|
||||
"Iter 153600, Minibatch Loss= 367.857788, Training Accuracy= 0.98438\n",
|
||||
"Iter 160000, Minibatch Loss= 1735.458740, Training Accuracy= 0.90625\n",
|
||||
"Iter 166400, Minibatch Loss= 209.320374, Training Accuracy= 0.95312\n",
|
||||
"Iter 172800, Minibatch Loss= 1788.553955, Training Accuracy= 0.90625\n",
|
||||
"Iter 179200, Minibatch Loss= 912.995544, Training Accuracy= 0.93750\n",
|
||||
"Iter 185600, Minibatch Loss= 2534.074463, Training Accuracy= 0.87500\n",
|
||||
"Iter 192000, Minibatch Loss= 73.052612, Training Accuracy= 0.96875\n",
|
||||
"Iter 198400, Minibatch Loss= 1609.606323, Training Accuracy= 0.93750\n",
|
||||
"Iter 204800, Minibatch Loss= 1823.219727, Training Accuracy= 0.96875\n",
|
||||
"Iter 211200, Minibatch Loss= 578.051086, Training Accuracy= 0.96875\n",
|
||||
"Iter 217600, Minibatch Loss= 1532.326172, Training Accuracy= 0.89062\n",
|
||||
"Iter 224000, Minibatch Loss= 769.775269, Training Accuracy= 0.95312\n",
|
||||
"Iter 230400, Minibatch Loss= 2614.737793, Training Accuracy= 0.92188\n",
|
||||
"Iter 236800, Minibatch Loss= 938.664368, Training Accuracy= 0.95312\n",
|
||||
"Iter 243200, Minibatch Loss= 1520.495605, Training Accuracy= 0.93750\n",
|
||||
"Iter 249600, Minibatch Loss= 657.419739, Training Accuracy= 0.95312\n",
|
||||
"Iter 256000, Minibatch Loss= 522.802124, Training Accuracy= 0.90625\n",
|
||||
"Iter 262400, Minibatch Loss= 211.188477, Training Accuracy= 0.96875\n",
|
||||
"Iter 268800, Minibatch Loss= 520.451172, Training Accuracy= 0.92188\n",
|
||||
"Iter 275200, Minibatch Loss= 1418.759155, Training Accuracy= 0.89062\n",
|
||||
"Iter 281600, Minibatch Loss= 241.748596, Training Accuracy= 0.96875\n",
|
||||
"Iter 288000, Minibatch Loss= 0.000000, Training Accuracy= 1.00000\n",
|
||||
"Iter 294400, Minibatch Loss= 1535.772827, Training Accuracy= 0.92188\n",
|
||||
"Optimization Finished!\n",
|
||||
"Testing Accuracy: 0.980469\n"
|
||||
]
|
||||
}
|
||||
],
|
||||
"source": [
|
||||
"# Launch the graph\n",
|
||||
"with tf.Session() as sess:\n",
|
||||
" sess.run(init)\n",
|
||||
" step = 1\n",
|
||||
" # Keep training until reach max iterations\n",
|
||||
" while step * batch_size < training_iters:\n",
|
||||
" batch_xs, batch_ys = mnist.train.next_batch(batch_size)\n",
|
||||
" # Fit training using batch data\n",
|
||||
" sess.run(optimizer, feed_dict={x: batch_xs, y: batch_ys, keep_prob: dropout})\n",
|
||||
" if step % display_step == 0:\n",
|
||||
" # Calculate batch accuracy\n",
|
||||
" acc = sess.run(accuracy, feed_dict={x: batch_xs, y: batch_ys, keep_prob: 1.})\n",
|
||||
" # Calculate batch loss\n",
|
||||
" loss = sess.run(cost, feed_dict={x: batch_xs, y: batch_ys, keep_prob: 1.})\n",
|
||||
" print \"Iter \" + str(step*batch_size) + \", Minibatch Loss= \" \\\n",
|
||||
" + \"{:.6f}\".format(loss) + \", Training Accuracy= \" + \"{:.5f}\".format(acc)\n",
|
||||
" step += 1\n",
|
||||
" print \"Optimization Finished!\"\n",
|
||||
" # Calculate accuracy for 256 mnist test images\n",
|
||||
" print \"Testing Accuracy:\", sess.run(accuracy, feed_dict={x: mnist.test.images[:256], \n",
|
||||
" y: mnist.test.labels[:256], \n",
|
||||
" keep_prob: 1.})"
|
||||
]
|
||||
}
|
||||
],
|
||||
"metadata": {
|
||||
"kernelspec": {
|
||||
"display_name": "Python 3",
|
||||
"language": "python",
|
||||
"name": "python3"
|
||||
},
|
||||
"language_info": {
|
||||
"codemirror_mode": {
|
||||
"name": "ipython",
|
||||
"version": 3
|
||||
},
|
||||
"file_extension": ".py",
|
||||
"mimetype": "text/x-python",
|
||||
"name": "python",
|
||||
"nbconvert_exporter": "python",
|
||||
"pygments_lexer": "ipython3",
|
||||
"version": "3.4.3"
|
||||
}
|
||||
},
|
||||
"nbformat": 4,
|
||||
"nbformat_minor": 0
|
||||
}
|
||||
|
|
@ -0,0 +1,321 @@
|
|||
{
|
||||
"cells": [
|
||||
{
|
||||
"cell_type": "markdown",
|
||||
"metadata": {},
|
||||
"source": [
|
||||
"# Convolutional Neural Network in TensorFlow\n",
|
||||
"\n",
|
||||
"Credits: Forked from [TensorFlow-Examples](https://github.com/aymericdamien/TensorFlow-Examples) by Aymeric Damien\n",
|
||||
"\n",
|
||||
"## Setup\n",
|
||||
"\n",
|
||||
"Refer to the [setup instructions](http://nbviewer.ipython.org/github/donnemartin/data-science-ipython-notebooks/blob/master/deep-learning/tensor-flow-examples/Setup_TensorFlow.md)"
|
||||
]
|
||||
},
|
||||
{
|
||||
"cell_type": "code",
|
||||
"execution_count": 2,
|
||||
"metadata": {
|
||||
"collapsed": false
|
||||
},
|
||||
"outputs": [
|
||||
{
|
||||
"name": "stdout",
|
||||
"output_type": "stream",
|
||||
"text": [
|
||||
"Extracting /tmp/data/train-images-idx3-ubyte.gz\n",
|
||||
"Extracting /tmp/data/train-labels-idx1-ubyte.gz\n",
|
||||
"Extracting /tmp/data/t10k-images-idx3-ubyte.gz\n",
|
||||
"Extracting /tmp/data/t10k-labels-idx1-ubyte.gz\n"
|
||||
]
|
||||
}
|
||||
],
|
||||
"source": [
|
||||
"# Import MINST data\n",
|
||||
"import input_data\n",
|
||||
"mnist = input_data.read_data_sets(\"/tmp/data/\", one_hot=True)"
|
||||
]
|
||||
},
|
||||
{
|
||||
"cell_type": "code",
|
||||
"execution_count": 3,
|
||||
"metadata": {
|
||||
"collapsed": true
|
||||
},
|
||||
"outputs": [],
|
||||
"source": [
|
||||
"import tensorflow as tf"
|
||||
]
|
||||
},
|
||||
{
|
||||
"cell_type": "code",
|
||||
"execution_count": 18,
|
||||
"metadata": {
|
||||
"collapsed": true
|
||||
},
|
||||
"outputs": [],
|
||||
"source": [
|
||||
"# Parameters\n",
|
||||
"learning_rate = 0.001\n",
|
||||
"training_iters = 100000\n",
|
||||
"batch_size = 128\n",
|
||||
"display_step = 20"
|
||||
]
|
||||
},
|
||||
{
|
||||
"cell_type": "code",
|
||||
"execution_count": 5,
|
||||
"metadata": {
|
||||
"collapsed": true
|
||||
},
|
||||
"outputs": [],
|
||||
"source": [
|
||||
"# Network Parameters\n",
|
||||
"n_input = 784 # MNIST data input (img shape: 28*28)\n",
|
||||
"n_classes = 10 # MNIST total classes (0-9 digits)\n",
|
||||
"dropout = 0.75 # Dropout, probability to keep units"
|
||||
]
|
||||
},
|
||||
{
|
||||
"cell_type": "code",
|
||||
"execution_count": 6,
|
||||
"metadata": {
|
||||
"collapsed": true
|
||||
},
|
||||
"outputs": [],
|
||||
"source": [
|
||||
"# tf Graph input\n",
|
||||
"x = tf.placeholder(tf.types.float32, [None, n_input])\n",
|
||||
"y = tf.placeholder(tf.types.float32, [None, n_classes])\n",
|
||||
"keep_prob = tf.placeholder(tf.types.float32) #dropout (keep probability)"
|
||||
]
|
||||
},
|
||||
{
|
||||
"cell_type": "code",
|
||||
"execution_count": 8,
|
||||
"metadata": {
|
||||
"collapsed": true
|
||||
},
|
||||
"outputs": [],
|
||||
"source": [
|
||||
"# Create model\n",
|
||||
"def conv2d(img, w, b):\n",
|
||||
" return tf.nn.relu(tf.nn.bias_add(tf.nn.conv2d(img, w, strides=[1, 1, 1, 1], \n",
|
||||
" padding='SAME'),b))\n",
|
||||
"\n",
|
||||
"def max_pool(img, k):\n",
|
||||
" return tf.nn.max_pool(img, ksize=[1, k, k, 1], strides=[1, k, k, 1], padding='SAME')\n",
|
||||
"\n",
|
||||
"def conv_net(_X, _weights, _biases, _dropout):\n",
|
||||
" # Reshape input picture\n",
|
||||
" _X = tf.reshape(_X, shape=[-1, 28, 28, 1])\n",
|
||||
"\n",
|
||||
" # Convolution Layer\n",
|
||||
" conv1 = conv2d(_X, _weights['wc1'], _biases['bc1'])\n",
|
||||
" # Max Pooling (down-sampling)\n",
|
||||
" conv1 = max_pool(conv1, k=2)\n",
|
||||
" # Apply Dropout\n",
|
||||
" conv1 = tf.nn.dropout(conv1, _dropout)\n",
|
||||
"\n",
|
||||
" # Convolution Layer\n",
|
||||
" conv2 = conv2d(conv1, _weights['wc2'], _biases['bc2'])\n",
|
||||
" # Max Pooling (down-sampling)\n",
|
||||
" conv2 = max_pool(conv2, k=2)\n",
|
||||
" # Apply Dropout\n",
|
||||
" conv2 = tf.nn.dropout(conv2, _dropout)\n",
|
||||
"\n",
|
||||
" # Fully connected layer\n",
|
||||
" # Reshape conv2 output to fit dense layer input\n",
|
||||
" dense1 = tf.reshape(conv2, [-1, _weights['wd1'].get_shape().as_list()[0]]) \n",
|
||||
" # Relu activation\n",
|
||||
" dense1 = tf.nn.relu(tf.add(tf.matmul(dense1, _weights['wd1']), _biases['bd1']))\n",
|
||||
" # Apply Dropout\n",
|
||||
" dense1 = tf.nn.dropout(dense1, _dropout) # Apply Dropout\n",
|
||||
"\n",
|
||||
" # Output, class prediction\n",
|
||||
" out = tf.add(tf.matmul(dense1, _weights['out']), _biases['out'])\n",
|
||||
" return out"
|
||||
]
|
||||
},
|
||||
{
|
||||
"cell_type": "code",
|
||||
"execution_count": 9,
|
||||
"metadata": {
|
||||
"collapsed": true
|
||||
},
|
||||
"outputs": [],
|
||||
"source": [
|
||||
"# Store layers weight & bias\n",
|
||||
"weights = {\n",
|
||||
" # 5x5 conv, 1 input, 32 outputs\n",
|
||||
" 'wc1': tf.Variable(tf.random_normal([5, 5, 1, 32])), \n",
|
||||
" # 5x5 conv, 32 inputs, 64 outputs\n",
|
||||
" 'wc2': tf.Variable(tf.random_normal([5, 5, 32, 64])), \n",
|
||||
" # fully connected, 7*7*64 inputs, 1024 outputs\n",
|
||||
" 'wd1': tf.Variable(tf.random_normal([7*7*64, 1024])), \n",
|
||||
" # 1024 inputs, 10 outputs (class prediction)\n",
|
||||
" 'out': tf.Variable(tf.random_normal([1024, n_classes])) \n",
|
||||
"}\n",
|
||||
"\n",
|
||||
"biases = {\n",
|
||||
" 'bc1': tf.Variable(tf.random_normal([32])),\n",
|
||||
" 'bc2': tf.Variable(tf.random_normal([64])),\n",
|
||||
" 'bd1': tf.Variable(tf.random_normal([1024])),\n",
|
||||
" 'out': tf.Variable(tf.random_normal([n_classes]))\n",
|
||||
"}"
|
||||
]
|
||||
},
|
||||
{
|
||||
"cell_type": "code",
|
||||
"execution_count": 10,
|
||||
"metadata": {
|
||||
"collapsed": true
|
||||
},
|
||||
"outputs": [],
|
||||
"source": [
|
||||
"# Construct model\n",
|
||||
"pred = conv_net(x, weights, biases, keep_prob)"
|
||||
]
|
||||
},
|
||||
{
|
||||
"cell_type": "code",
|
||||
"execution_count": 11,
|
||||
"metadata": {
|
||||
"collapsed": true
|
||||
},
|
||||
"outputs": [],
|
||||
"source": [
|
||||
"# Define loss and optimizer\n",
|
||||
"cost = tf.reduce_mean(tf.nn.softmax_cross_entropy_with_logits(pred, y))\n",
|
||||
"optimizer = tf.train.AdamOptimizer(learning_rate=learning_rate).minimize(cost)"
|
||||
]
|
||||
},
|
||||
{
|
||||
"cell_type": "code",
|
||||
"execution_count": 12,
|
||||
"metadata": {
|
||||
"collapsed": true
|
||||
},
|
||||
"outputs": [],
|
||||
"source": [
|
||||
"# Evaluate model\n",
|
||||
"correct_pred = tf.equal(tf.argmax(pred,1), tf.argmax(y,1))\n",
|
||||
"accuracy = tf.reduce_mean(tf.cast(correct_pred, tf.types.float32))"
|
||||
]
|
||||
},
|
||||
{
|
||||
"cell_type": "code",
|
||||
"execution_count": 13,
|
||||
"metadata": {
|
||||
"collapsed": true
|
||||
},
|
||||
"outputs": [],
|
||||
"source": [
|
||||
"# Initializing the variables\n",
|
||||
"init = tf.initialize_all_variables()"
|
||||
]
|
||||
},
|
||||
{
|
||||
"cell_type": "code",
|
||||
"execution_count": 19,
|
||||
"metadata": {
|
||||
"collapsed": false
|
||||
},
|
||||
"outputs": [
|
||||
{
|
||||
"name": "stdout",
|
||||
"output_type": "stream",
|
||||
"text": [
|
||||
"Iter 2560, Minibatch Loss= 26046.011719, Training Accuracy= 0.21094\n",
|
||||
"Iter 5120, Minibatch Loss= 10456.769531, Training Accuracy= 0.52344\n",
|
||||
"Iter 7680, Minibatch Loss= 6273.207520, Training Accuracy= 0.71875\n",
|
||||
"Iter 10240, Minibatch Loss= 6276.231445, Training Accuracy= 0.64062\n",
|
||||
"Iter 12800, Minibatch Loss= 4188.221680, Training Accuracy= 0.77344\n",
|
||||
"Iter 15360, Minibatch Loss= 2717.077637, Training Accuracy= 0.80469\n",
|
||||
"Iter 17920, Minibatch Loss= 4057.120361, Training Accuracy= 0.81250\n",
|
||||
"Iter 20480, Minibatch Loss= 1696.550415, Training Accuracy= 0.87500\n",
|
||||
"Iter 23040, Minibatch Loss= 2525.317627, Training Accuracy= 0.85938\n",
|
||||
"Iter 25600, Minibatch Loss= 2341.906738, Training Accuracy= 0.87500\n",
|
||||
"Iter 28160, Minibatch Loss= 4200.535156, Training Accuracy= 0.79688\n",
|
||||
"Iter 30720, Minibatch Loss= 1888.964355, Training Accuracy= 0.89062\n",
|
||||
"Iter 33280, Minibatch Loss= 2167.645996, Training Accuracy= 0.84375\n",
|
||||
"Iter 35840, Minibatch Loss= 1932.107544, Training Accuracy= 0.89844\n",
|
||||
"Iter 38400, Minibatch Loss= 1562.430054, Training Accuracy= 0.90625\n",
|
||||
"Iter 40960, Minibatch Loss= 1676.755249, Training Accuracy= 0.84375\n",
|
||||
"Iter 43520, Minibatch Loss= 1003.626099, Training Accuracy= 0.93750\n",
|
||||
"Iter 46080, Minibatch Loss= 1176.615479, Training Accuracy= 0.86719\n",
|
||||
"Iter 48640, Minibatch Loss= 1260.592651, Training Accuracy= 0.88281\n",
|
||||
"Iter 51200, Minibatch Loss= 1399.667969, Training Accuracy= 0.86719\n",
|
||||
"Iter 53760, Minibatch Loss= 1259.961426, Training Accuracy= 0.89844\n",
|
||||
"Iter 56320, Minibatch Loss= 1415.800781, Training Accuracy= 0.89062\n",
|
||||
"Iter 58880, Minibatch Loss= 1835.365967, Training Accuracy= 0.85156\n",
|
||||
"Iter 61440, Minibatch Loss= 1395.168823, Training Accuracy= 0.90625\n",
|
||||
"Iter 64000, Minibatch Loss= 973.283569, Training Accuracy= 0.88281\n",
|
||||
"Iter 66560, Minibatch Loss= 818.093811, Training Accuracy= 0.92969\n",
|
||||
"Iter 69120, Minibatch Loss= 1178.744263, Training Accuracy= 0.92188\n",
|
||||
"Iter 71680, Minibatch Loss= 845.889709, Training Accuracy= 0.89844\n",
|
||||
"Iter 74240, Minibatch Loss= 1259.505615, Training Accuracy= 0.90625\n",
|
||||
"Iter 76800, Minibatch Loss= 738.037109, Training Accuracy= 0.89844\n",
|
||||
"Iter 79360, Minibatch Loss= 862.499146, Training Accuracy= 0.93750\n",
|
||||
"Iter 81920, Minibatch Loss= 739.704041, Training Accuracy= 0.90625\n",
|
||||
"Iter 84480, Minibatch Loss= 652.880310, Training Accuracy= 0.95312\n",
|
||||
"Iter 87040, Minibatch Loss= 635.464600, Training Accuracy= 0.92969\n",
|
||||
"Iter 89600, Minibatch Loss= 933.166626, Training Accuracy= 0.90625\n",
|
||||
"Iter 92160, Minibatch Loss= 213.874893, Training Accuracy= 0.96094\n",
|
||||
"Iter 94720, Minibatch Loss= 609.575684, Training Accuracy= 0.91406\n",
|
||||
"Iter 97280, Minibatch Loss= 560.208008, Training Accuracy= 0.93750\n",
|
||||
"Iter 99840, Minibatch Loss= 963.577148, Training Accuracy= 0.90625\n",
|
||||
"Optimization Finished!\n",
|
||||
"Testing Accuracy: 0.960938\n"
|
||||
]
|
||||
}
|
||||
],
|
||||
"source": [
|
||||
"# Launch the graph\n",
|
||||
"with tf.Session() as sess:\n",
|
||||
" sess.run(init)\n",
|
||||
" step = 1\n",
|
||||
" # Keep training until reach max iterations\n",
|
||||
" while step * batch_size < training_iters:\n",
|
||||
" batch_xs, batch_ys = mnist.train.next_batch(batch_size)\n",
|
||||
" # Fit training using batch data\n",
|
||||
" sess.run(optimizer, feed_dict={x: batch_xs, y: batch_ys, keep_prob: dropout})\n",
|
||||
" if step % display_step == 0:\n",
|
||||
" # Calculate batch accuracy\n",
|
||||
" acc = sess.run(accuracy, feed_dict={x: batch_xs, y: batch_ys, keep_prob: 1.})\n",
|
||||
" # Calculate batch loss\n",
|
||||
" loss = sess.run(cost, feed_dict={x: batch_xs, y: batch_ys, keep_prob: 1.})\n",
|
||||
" print \"Iter \" + str(step*batch_size) + \", Minibatch Loss= \" + \\\n",
|
||||
" \"{:.6f}\".format(loss) + \", Training Accuracy= \" + \"{:.5f}\".format(acc)\n",
|
||||
" step += 1\n",
|
||||
" print \"Optimization Finished!\"\n",
|
||||
" # Calculate accuracy for 256 mnist test images\n",
|
||||
" print \"Testing Accuracy:\", sess.run(accuracy, feed_dict={x: mnist.test.images[:256], \n",
|
||||
" y: mnist.test.labels[:256], \n",
|
||||
" keep_prob: 1.})"
|
||||
]
|
||||
}
|
||||
],
|
||||
"metadata": {
|
||||
"kernelspec": {
|
||||
"display_name": "Python 3",
|
||||
"language": "python",
|
||||
"name": "python3"
|
||||
},
|
||||
"language_info": {
|
||||
"codemirror_mode": {
|
||||
"name": "ipython",
|
||||
"version": 3
|
||||
},
|
||||
"file_extension": ".py",
|
||||
"mimetype": "text/x-python",
|
||||
"name": "python",
|
||||
"nbconvert_exporter": "python",
|
||||
"pygments_lexer": "ipython3",
|
||||
"version": "3.4.3"
|
||||
}
|
||||
},
|
||||
"nbformat": 4,
|
||||
"nbformat_minor": 0
|
||||
}
|
||||
|
|
@ -0,0 +1,253 @@
|
|||
{
|
||||
"cells": [
|
||||
{
|
||||
"cell_type": "markdown",
|
||||
"metadata": {},
|
||||
"source": [
|
||||
"# Multilayer Perceptron in TensorFlow\n",
|
||||
"\n",
|
||||
"Credits: Forked from [TensorFlow-Examples](https://github.com/aymericdamien/TensorFlow-Examples) by Aymeric Damien\n",
|
||||
"\n",
|
||||
"## Setup\n",
|
||||
"\n",
|
||||
"Refer to the [setup instructions](http://nbviewer.ipython.org/github/donnemartin/data-science-ipython-notebooks/blob/master/deep-learning/tensor-flow-examples/Setup_TensorFlow.md)"
|
||||
]
|
||||
},
|
||||
{
|
||||
"cell_type": "code",
|
||||
"execution_count": 2,
|
||||
"metadata": {
|
||||
"collapsed": false
|
||||
},
|
||||
"outputs": [
|
||||
{
|
||||
"name": "stdout",
|
||||
"output_type": "stream",
|
||||
"text": [
|
||||
"Extracting /tmp/data/train-images-idx3-ubyte.gz\n",
|
||||
"Extracting /tmp/data/train-labels-idx1-ubyte.gz\n",
|
||||
"Extracting /tmp/data/t10k-images-idx3-ubyte.gz\n",
|
||||
"Extracting /tmp/data/t10k-labels-idx1-ubyte.gz\n"
|
||||
]
|
||||
}
|
||||
],
|
||||
"source": [
|
||||
"# Import MINST data\n",
|
||||
"import input_data\n",
|
||||
"mnist = input_data.read_data_sets(\"/tmp/data/\", one_hot=True)"
|
||||
]
|
||||
},
|
||||
{
|
||||
"cell_type": "code",
|
||||
"execution_count": 3,
|
||||
"metadata": {
|
||||
"collapsed": true
|
||||
},
|
||||
"outputs": [],
|
||||
"source": [
|
||||
"import tensorflow as tf"
|
||||
]
|
||||
},
|
||||
{
|
||||
"cell_type": "code",
|
||||
"execution_count": 4,
|
||||
"metadata": {
|
||||
"collapsed": true
|
||||
},
|
||||
"outputs": [],
|
||||
"source": [
|
||||
"# Parameters\n",
|
||||
"learning_rate = 0.001\n",
|
||||
"training_epochs = 15\n",
|
||||
"batch_size = 100\n",
|
||||
"display_step = 1"
|
||||
]
|
||||
},
|
||||
{
|
||||
"cell_type": "code",
|
||||
"execution_count": 5,
|
||||
"metadata": {
|
||||
"collapsed": true
|
||||
},
|
||||
"outputs": [],
|
||||
"source": [
|
||||
"# Network Parameters\n",
|
||||
"n_hidden_1 = 256 # 1st layer num features\n",
|
||||
"n_hidden_2 = 256 # 2nd layer num features\n",
|
||||
"n_input = 784 # MNIST data input (img shape: 28*28)\n",
|
||||
"n_classes = 10 # MNIST total classes (0-9 digits)"
|
||||
]
|
||||
},
|
||||
{
|
||||
"cell_type": "code",
|
||||
"execution_count": 6,
|
||||
"metadata": {
|
||||
"collapsed": true
|
||||
},
|
||||
"outputs": [],
|
||||
"source": [
|
||||
"# tf Graph input\n",
|
||||
"x = tf.placeholder(\"float\", [None, n_input])\n",
|
||||
"y = tf.placeholder(\"float\", [None, n_classes])"
|
||||
]
|
||||
},
|
||||
{
|
||||
"cell_type": "code",
|
||||
"execution_count": 7,
|
||||
"metadata": {
|
||||
"collapsed": true
|
||||
},
|
||||
"outputs": [],
|
||||
"source": [
|
||||
"# Create model\n",
|
||||
"def multilayer_perceptron(_X, _weights, _biases):\n",
|
||||
" #Hidden layer with RELU activation\n",
|
||||
" layer_1 = tf.nn.relu(tf.add(tf.matmul(_X, _weights['h1']), _biases['b1'])) \n",
|
||||
" #Hidden layer with RELU activation\n",
|
||||
" layer_2 = tf.nn.relu(tf.add(tf.matmul(layer_1, _weights['h2']), _biases['b2'])) \n",
|
||||
" return tf.matmul(layer_2, weights['out']) + biases['out']"
|
||||
]
|
||||
},
|
||||
{
|
||||
"cell_type": "code",
|
||||
"execution_count": 8,
|
||||
"metadata": {
|
||||
"collapsed": true
|
||||
},
|
||||
"outputs": [],
|
||||
"source": [
|
||||
"# Store layers weight & bias\n",
|
||||
"weights = {\n",
|
||||
" 'h1': tf.Variable(tf.random_normal([n_input, n_hidden_1])),\n",
|
||||
" 'h2': tf.Variable(tf.random_normal([n_hidden_1, n_hidden_2])),\n",
|
||||
" 'out': tf.Variable(tf.random_normal([n_hidden_2, n_classes]))\n",
|
||||
"}\n",
|
||||
"biases = {\n",
|
||||
" 'b1': tf.Variable(tf.random_normal([n_hidden_1])),\n",
|
||||
" 'b2': tf.Variable(tf.random_normal([n_hidden_2])),\n",
|
||||
" 'out': tf.Variable(tf.random_normal([n_classes]))\n",
|
||||
"}"
|
||||
]
|
||||
},
|
||||
{
|
||||
"cell_type": "code",
|
||||
"execution_count": 9,
|
||||
"metadata": {
|
||||
"collapsed": true
|
||||
},
|
||||
"outputs": [],
|
||||
"source": [
|
||||
"# Construct model\n",
|
||||
"pred = multilayer_perceptron(x, weights, biases)"
|
||||
]
|
||||
},
|
||||
{
|
||||
"cell_type": "code",
|
||||
"execution_count": 10,
|
||||
"metadata": {
|
||||
"collapsed": true
|
||||
},
|
||||
"outputs": [],
|
||||
"source": [
|
||||
"# Define loss and optimizer\n",
|
||||
"# Softmax loss\n",
|
||||
"cost = tf.reduce_mean(tf.nn.softmax_cross_entropy_with_logits(pred, y)) \n",
|
||||
"# Adam Optimizer\n",
|
||||
"optimizer = tf.train.AdamOptimizer(learning_rate=learning_rate).minimize(cost) "
|
||||
]
|
||||
},
|
||||
{
|
||||
"cell_type": "code",
|
||||
"execution_count": 11,
|
||||
"metadata": {
|
||||
"collapsed": true
|
||||
},
|
||||
"outputs": [],
|
||||
"source": [
|
||||
"# Initializing the variables\n",
|
||||
"init = tf.initialize_all_variables()"
|
||||
]
|
||||
},
|
||||
{
|
||||
"cell_type": "code",
|
||||
"execution_count": 12,
|
||||
"metadata": {
|
||||
"collapsed": false
|
||||
},
|
||||
"outputs": [
|
||||
{
|
||||
"name": "stdout",
|
||||
"output_type": "stream",
|
||||
"text": [
|
||||
"Epoch: 0001 cost= 160.113980416\n",
|
||||
"Epoch: 0002 cost= 38.665780694\n",
|
||||
"Epoch: 0003 cost= 24.118004577\n",
|
||||
"Epoch: 0004 cost= 16.440921303\n",
|
||||
"Epoch: 0005 cost= 11.689460141\n",
|
||||
"Epoch: 0006 cost= 8.469423468\n",
|
||||
"Epoch: 0007 cost= 6.223237230\n",
|
||||
"Epoch: 0008 cost= 4.560174118\n",
|
||||
"Epoch: 0009 cost= 3.250516910\n",
|
||||
"Epoch: 0010 cost= 2.359658795\n",
|
||||
"Epoch: 0011 cost= 1.694081847\n",
|
||||
"Epoch: 0012 cost= 1.167997509\n",
|
||||
"Epoch: 0013 cost= 0.872986831\n",
|
||||
"Epoch: 0014 cost= 0.630616366\n",
|
||||
"Epoch: 0015 cost= 0.487381571\n",
|
||||
"Optimization Finished!\n",
|
||||
"Accuracy: 0.9462\n"
|
||||
]
|
||||
}
|
||||
],
|
||||
"source": [
|
||||
"# Launch the graph\n",
|
||||
"with tf.Session() as sess:\n",
|
||||
" sess.run(init)\n",
|
||||
"\n",
|
||||
" # Training cycle\n",
|
||||
" for epoch in range(training_epochs):\n",
|
||||
" avg_cost = 0.\n",
|
||||
" total_batch = int(mnist.train.num_examples/batch_size)\n",
|
||||
" # Loop over all batches\n",
|
||||
" for i in range(total_batch):\n",
|
||||
" batch_xs, batch_ys = mnist.train.next_batch(batch_size)\n",
|
||||
" # Fit training using batch data\n",
|
||||
" sess.run(optimizer, feed_dict={x: batch_xs, y: batch_ys})\n",
|
||||
" # Compute average loss\n",
|
||||
" avg_cost += sess.run(cost, feed_dict={x: batch_xs, y: batch_ys})/total_batch\n",
|
||||
" # Display logs per epoch step\n",
|
||||
" if epoch % display_step == 0:\n",
|
||||
" print \"Epoch:\", '%04d' % (epoch+1), \"cost=\", \"{:.9f}\".format(avg_cost)\n",
|
||||
"\n",
|
||||
" print \"Optimization Finished!\"\n",
|
||||
"\n",
|
||||
" # Test model\n",
|
||||
" correct_prediction = tf.equal(tf.argmax(pred, 1), tf.argmax(y, 1))\n",
|
||||
" # Calculate accuracy\n",
|
||||
" accuracy = tf.reduce_mean(tf.cast(correct_prediction, \"float\"))\n",
|
||||
" print \"Accuracy:\", accuracy.eval({x: mnist.test.images, y: mnist.test.labels})"
|
||||
]
|
||||
}
|
||||
],
|
||||
"metadata": {
|
||||
"kernelspec": {
|
||||
"display_name": "Python 3",
|
||||
"language": "python",
|
||||
"name": "python3"
|
||||
},
|
||||
"language_info": {
|
||||
"codemirror_mode": {
|
||||
"name": "ipython",
|
||||
"version": 3
|
||||
},
|
||||
"file_extension": ".py",
|
||||
"mimetype": "text/x-python",
|
||||
"name": "python",
|
||||
"nbconvert_exporter": "python",
|
||||
"pygments_lexer": "ipython3",
|
||||
"version": "3.4.3"
|
||||
}
|
||||
},
|
||||
"nbformat": 4,
|
||||
"nbformat_minor": 0
|
||||
}
|
||||
|
|
@ -0,0 +1,294 @@
|
|||
{
|
||||
"cells": [
|
||||
{
|
||||
"cell_type": "markdown",
|
||||
"metadata": {},
|
||||
"source": [
|
||||
"# Recurrent Neural Network in TensorFlow\n",
|
||||
"\n",
|
||||
"Credits: Forked from [TensorFlow-Examples](https://github.com/aymericdamien/TensorFlow-Examples) by Aymeric Damien\n",
|
||||
"\n",
|
||||
"## Setup\n",
|
||||
"\n",
|
||||
"Refer to the [setup instructions](http://nbviewer.ipython.org/github/donnemartin/data-science-ipython-notebooks/blob/master/deep-learning/tensor-flow-examples/Setup_TensorFlow.md)"
|
||||
]
|
||||
},
|
||||
{
|
||||
"cell_type": "code",
|
||||
"execution_count": 2,
|
||||
"metadata": {
|
||||
"collapsed": false
|
||||
},
|
||||
"outputs": [
|
||||
{
|
||||
"name": "stdout",
|
||||
"output_type": "stream",
|
||||
"text": [
|
||||
"Extracting /tmp/data/train-images-idx3-ubyte.gz\n",
|
||||
"Extracting /tmp/data/train-labels-idx1-ubyte.gz\n",
|
||||
"Extracting /tmp/data/t10k-images-idx3-ubyte.gz\n",
|
||||
"Extracting /tmp/data/t10k-labels-idx1-ubyte.gz\n"
|
||||
]
|
||||
}
|
||||
],
|
||||
"source": [
|
||||
"# Import MINST data\n",
|
||||
"import input_data\n",
|
||||
"mnist = input_data.read_data_sets(\"/tmp/data/\", one_hot=True)\n",
|
||||
"\n",
|
||||
"import tensorflow as tf\n",
|
||||
"from tensorflow.models.rnn import rnn, rnn_cell\n",
|
||||
"import numpy as np"
|
||||
]
|
||||
},
|
||||
{
|
||||
"cell_type": "code",
|
||||
"execution_count": 3,
|
||||
"metadata": {
|
||||
"collapsed": true
|
||||
},
|
||||
"outputs": [],
|
||||
"source": [
|
||||
"'''\n",
|
||||
"To classify images using a reccurent neural network, we consider every image row as a sequence of pixels.\n",
|
||||
"Because MNIST image shape is 28*28px, we will then handle 28 sequences of 28 steps for every sample.\n",
|
||||
"'''\n",
|
||||
"\n",
|
||||
"# Parameters\n",
|
||||
"learning_rate = 0.001\n",
|
||||
"training_iters = 100000\n",
|
||||
"batch_size = 128\n",
|
||||
"display_step = 10\n",
|
||||
"\n",
|
||||
"# Network Parameters\n",
|
||||
"n_input = 28 # MNIST data input (img shape: 28*28)\n",
|
||||
"n_steps = 28 # timesteps\n",
|
||||
"n_hidden = 128 # hidden layer num of features\n",
|
||||
"n_classes = 10 # MNIST total classes (0-9 digits)"
|
||||
]
|
||||
},
|
||||
{
|
||||
"cell_type": "code",
|
||||
"execution_count": 4,
|
||||
"metadata": {
|
||||
"collapsed": true
|
||||
},
|
||||
"outputs": [],
|
||||
"source": [
|
||||
"# tf Graph input\n",
|
||||
"x = tf.placeholder(\"float\", [None, n_steps, n_input])\n",
|
||||
"istate = tf.placeholder(\"float\", [None, 2*n_hidden]) #state & cell => 2x n_hidden\n",
|
||||
"y = tf.placeholder(\"float\", [None, n_classes])\n",
|
||||
"\n",
|
||||
"# Define weights\n",
|
||||
"weights = {\n",
|
||||
" 'hidden': tf.Variable(tf.random_normal([n_input, n_hidden])), # Hidden layer weights\n",
|
||||
" 'out': tf.Variable(tf.random_normal([n_hidden, n_classes]))\n",
|
||||
"}\n",
|
||||
"biases = {\n",
|
||||
" 'hidden': tf.Variable(tf.random_normal([n_hidden])),\n",
|
||||
" 'out': tf.Variable(tf.random_normal([n_classes]))\n",
|
||||
"}"
|
||||
]
|
||||
},
|
||||
{
|
||||
"cell_type": "code",
|
||||
"execution_count": 5,
|
||||
"metadata": {
|
||||
"collapsed": true
|
||||
},
|
||||
"outputs": [],
|
||||
"source": [
|
||||
"def RNN(_X, _istate, _weights, _biases):\n",
|
||||
"\n",
|
||||
" # input shape: (batch_size, n_steps, n_input)\n",
|
||||
" _X = tf.transpose(_X, [1, 0, 2]) # permute n_steps and batch_size\n",
|
||||
" # Reshape to prepare input to hidden activation\n",
|
||||
" _X = tf.reshape(_X, [-1, n_input]) # (n_steps*batch_size, n_input)\n",
|
||||
" # Linear activation\n",
|
||||
" _X = tf.matmul(_X, _weights['hidden']) + _biases['hidden']\n",
|
||||
"\n",
|
||||
" # Define a lstm cell with tensorflow\n",
|
||||
" lstm_cell = rnn_cell.BasicLSTMCell(n_hidden, forget_bias=1.0)\n",
|
||||
" # Split data because rnn cell needs a list of inputs for the RNN inner loop\n",
|
||||
" _X = tf.split(0, n_steps, _X) # n_steps * (batch_size, n_hidden)\n",
|
||||
"\n",
|
||||
" # Get lstm cell output\n",
|
||||
" outputs, states = rnn.rnn(lstm_cell, _X, initial_state=_istate)\n",
|
||||
"\n",
|
||||
" # Linear activation\n",
|
||||
" # Get inner loop last output\n",
|
||||
" return tf.matmul(outputs[-1], _weights['out']) + _biases['out']"
|
||||
]
|
||||
},
|
||||
{
|
||||
"cell_type": "code",
|
||||
"execution_count": 6,
|
||||
"metadata": {
|
||||
"collapsed": false
|
||||
},
|
||||
"outputs": [],
|
||||
"source": [
|
||||
"pred = RNN(x, istate, weights, biases)\n",
|
||||
"\n",
|
||||
"# Define loss and optimizer\n",
|
||||
"cost = tf.reduce_mean(tf.nn.softmax_cross_entropy_with_logits(pred, y)) # Softmax loss\n",
|
||||
"optimizer = tf.train.AdamOptimizer(learning_rate=learning_rate).minimize(cost) # Adam Optimizer\n",
|
||||
"\n",
|
||||
"# Evaluate model\n",
|
||||
"correct_pred = tf.equal(tf.argmax(pred,1), tf.argmax(y,1))\n",
|
||||
"accuracy = tf.reduce_mean(tf.cast(correct_pred, tf.types.float32))"
|
||||
]
|
||||
},
|
||||
{
|
||||
"cell_type": "code",
|
||||
"execution_count": 7,
|
||||
"metadata": {
|
||||
"collapsed": false
|
||||
},
|
||||
"outputs": [
|
||||
{
|
||||
"name": "stdout",
|
||||
"output_type": "stream",
|
||||
"text": [
|
||||
"Iter 1280, Minibatch Loss= 1.888242, Training Accuracy= 0.39844\n",
|
||||
"Iter 2560, Minibatch Loss= 1.519879, Training Accuracy= 0.47656\n",
|
||||
"Iter 3840, Minibatch Loss= 1.238005, Training Accuracy= 0.63281\n",
|
||||
"Iter 5120, Minibatch Loss= 0.933760, Training Accuracy= 0.71875\n",
|
||||
"Iter 6400, Minibatch Loss= 0.832130, Training Accuracy= 0.73438\n",
|
||||
"Iter 7680, Minibatch Loss= 0.979760, Training Accuracy= 0.70312\n",
|
||||
"Iter 8960, Minibatch Loss= 0.821921, Training Accuracy= 0.71875\n",
|
||||
"Iter 10240, Minibatch Loss= 0.710566, Training Accuracy= 0.79688\n",
|
||||
"Iter 11520, Minibatch Loss= 0.578501, Training Accuracy= 0.82812\n",
|
||||
"Iter 12800, Minibatch Loss= 0.765049, Training Accuracy= 0.75000\n",
|
||||
"Iter 14080, Minibatch Loss= 0.582995, Training Accuracy= 0.78125\n",
|
||||
"Iter 15360, Minibatch Loss= 0.575092, Training Accuracy= 0.79688\n",
|
||||
"Iter 16640, Minibatch Loss= 0.701214, Training Accuracy= 0.75781\n",
|
||||
"Iter 17920, Minibatch Loss= 0.561972, Training Accuracy= 0.78125\n",
|
||||
"Iter 19200, Minibatch Loss= 0.394480, Training Accuracy= 0.85938\n",
|
||||
"Iter 20480, Minibatch Loss= 0.356244, Training Accuracy= 0.91406\n",
|
||||
"Iter 21760, Minibatch Loss= 0.632163, Training Accuracy= 0.78125\n",
|
||||
"Iter 23040, Minibatch Loss= 0.269334, Training Accuracy= 0.90625\n",
|
||||
"Iter 24320, Minibatch Loss= 0.485007, Training Accuracy= 0.86719\n",
|
||||
"Iter 25600, Minibatch Loss= 0.569704, Training Accuracy= 0.78906\n",
|
||||
"Iter 26880, Minibatch Loss= 0.267697, Training Accuracy= 0.92188\n",
|
||||
"Iter 28160, Minibatch Loss= 0.381177, Training Accuracy= 0.90625\n",
|
||||
"Iter 29440, Minibatch Loss= 0.350800, Training Accuracy= 0.87500\n",
|
||||
"Iter 30720, Minibatch Loss= 0.356782, Training Accuracy= 0.90625\n",
|
||||
"Iter 32000, Minibatch Loss= 0.322511, Training Accuracy= 0.89062\n",
|
||||
"Iter 33280, Minibatch Loss= 0.309195, Training Accuracy= 0.90625\n",
|
||||
"Iter 34560, Minibatch Loss= 0.535408, Training Accuracy= 0.83594\n",
|
||||
"Iter 35840, Minibatch Loss= 0.281643, Training Accuracy= 0.92969\n",
|
||||
"Iter 37120, Minibatch Loss= 0.290962, Training Accuracy= 0.89844\n",
|
||||
"Iter 38400, Minibatch Loss= 0.204718, Training Accuracy= 0.93750\n",
|
||||
"Iter 39680, Minibatch Loss= 0.205882, Training Accuracy= 0.92969\n",
|
||||
"Iter 40960, Minibatch Loss= 0.481441, Training Accuracy= 0.84375\n",
|
||||
"Iter 42240, Minibatch Loss= 0.348245, Training Accuracy= 0.89844\n",
|
||||
"Iter 43520, Minibatch Loss= 0.274692, Training Accuracy= 0.90625\n",
|
||||
"Iter 44800, Minibatch Loss= 0.171815, Training Accuracy= 0.94531\n",
|
||||
"Iter 46080, Minibatch Loss= 0.171035, Training Accuracy= 0.93750\n",
|
||||
"Iter 47360, Minibatch Loss= 0.235800, Training Accuracy= 0.89844\n",
|
||||
"Iter 48640, Minibatch Loss= 0.235974, Training Accuracy= 0.93750\n",
|
||||
"Iter 49920, Minibatch Loss= 0.207323, Training Accuracy= 0.92188\n",
|
||||
"Iter 51200, Minibatch Loss= 0.212989, Training Accuracy= 0.91406\n",
|
||||
"Iter 52480, Minibatch Loss= 0.151774, Training Accuracy= 0.95312\n",
|
||||
"Iter 53760, Minibatch Loss= 0.090070, Training Accuracy= 0.96875\n",
|
||||
"Iter 55040, Minibatch Loss= 0.264714, Training Accuracy= 0.92969\n",
|
||||
"Iter 56320, Minibatch Loss= 0.235086, Training Accuracy= 0.92969\n",
|
||||
"Iter 57600, Minibatch Loss= 0.160302, Training Accuracy= 0.95312\n",
|
||||
"Iter 58880, Minibatch Loss= 0.106515, Training Accuracy= 0.96875\n",
|
||||
"Iter 60160, Minibatch Loss= 0.236039, Training Accuracy= 0.94531\n",
|
||||
"Iter 61440, Minibatch Loss= 0.279540, Training Accuracy= 0.90625\n",
|
||||
"Iter 62720, Minibatch Loss= 0.173585, Training Accuracy= 0.93750\n",
|
||||
"Iter 64000, Minibatch Loss= 0.191009, Training Accuracy= 0.92188\n",
|
||||
"Iter 65280, Minibatch Loss= 0.210331, Training Accuracy= 0.89844\n",
|
||||
"Iter 66560, Minibatch Loss= 0.223444, Training Accuracy= 0.94531\n",
|
||||
"Iter 67840, Minibatch Loss= 0.278210, Training Accuracy= 0.91406\n",
|
||||
"Iter 69120, Minibatch Loss= 0.174290, Training Accuracy= 0.95312\n",
|
||||
"Iter 70400, Minibatch Loss= 0.188701, Training Accuracy= 0.94531\n",
|
||||
"Iter 71680, Minibatch Loss= 0.210277, Training Accuracy= 0.94531\n",
|
||||
"Iter 72960, Minibatch Loss= 0.249951, Training Accuracy= 0.95312\n",
|
||||
"Iter 74240, Minibatch Loss= 0.209853, Training Accuracy= 0.92188\n",
|
||||
"Iter 75520, Minibatch Loss= 0.049742, Training Accuracy= 0.99219\n",
|
||||
"Iter 76800, Minibatch Loss= 0.250095, Training Accuracy= 0.92969\n",
|
||||
"Iter 78080, Minibatch Loss= 0.133853, Training Accuracy= 0.95312\n",
|
||||
"Iter 79360, Minibatch Loss= 0.110206, Training Accuracy= 0.97656\n",
|
||||
"Iter 80640, Minibatch Loss= 0.141906, Training Accuracy= 0.93750\n",
|
||||
"Iter 81920, Minibatch Loss= 0.126872, Training Accuracy= 0.94531\n",
|
||||
"Iter 83200, Minibatch Loss= 0.138925, Training Accuracy= 0.95312\n",
|
||||
"Iter 84480, Minibatch Loss= 0.128652, Training Accuracy= 0.96094\n",
|
||||
"Iter 85760, Minibatch Loss= 0.099837, Training Accuracy= 0.96094\n",
|
||||
"Iter 87040, Minibatch Loss= 0.119000, Training Accuracy= 0.95312\n",
|
||||
"Iter 88320, Minibatch Loss= 0.179807, Training Accuracy= 0.95312\n",
|
||||
"Iter 89600, Minibatch Loss= 0.141792, Training Accuracy= 0.96094\n",
|
||||
"Iter 90880, Minibatch Loss= 0.142424, Training Accuracy= 0.96094\n",
|
||||
"Iter 92160, Minibatch Loss= 0.159564, Training Accuracy= 0.96094\n",
|
||||
"Iter 93440, Minibatch Loss= 0.111984, Training Accuracy= 0.95312\n",
|
||||
"Iter 94720, Minibatch Loss= 0.238978, Training Accuracy= 0.92969\n",
|
||||
"Iter 96000, Minibatch Loss= 0.068002, Training Accuracy= 0.97656\n",
|
||||
"Iter 97280, Minibatch Loss= 0.191819, Training Accuracy= 0.94531\n",
|
||||
"Iter 98560, Minibatch Loss= 0.081197, Training Accuracy= 0.99219\n",
|
||||
"Iter 99840, Minibatch Loss= 0.206797, Training Accuracy= 0.95312\n",
|
||||
"Optimization Finished!\n",
|
||||
"Testing Accuracy: 0.941406\n"
|
||||
]
|
||||
}
|
||||
],
|
||||
"source": [
|
||||
"# Initializing the variables\n",
|
||||
"init = tf.initialize_all_variables()\n",
|
||||
"\n",
|
||||
"# Launch the graph\n",
|
||||
"with tf.Session() as sess:\n",
|
||||
" sess.run(init)\n",
|
||||
" step = 1\n",
|
||||
" # Keep training until reach max iterations\n",
|
||||
" while step * batch_size < training_iters:\n",
|
||||
" batch_xs, batch_ys = mnist.train.next_batch(batch_size)\n",
|
||||
" # Reshape data to get 28 seq of 28 elements\n",
|
||||
" batch_xs = batch_xs.reshape((batch_size, n_steps, n_input))\n",
|
||||
" # Fit training using batch data\n",
|
||||
" sess.run(optimizer, feed_dict={x: batch_xs, y: batch_ys,\n",
|
||||
" istate: np.zeros((batch_size, 2*n_hidden))})\n",
|
||||
" if step % display_step == 0:\n",
|
||||
" # Calculate batch accuracy\n",
|
||||
" acc = sess.run(accuracy, feed_dict={x: batch_xs, y: batch_ys,\n",
|
||||
" istate: np.zeros((batch_size, 2*n_hidden))})\n",
|
||||
" # Calculate batch loss\n",
|
||||
" loss = sess.run(cost, feed_dict={x: batch_xs, y: batch_ys,\n",
|
||||
" istate: np.zeros((batch_size, 2*n_hidden))})\n",
|
||||
" print \"Iter \" + str(step*batch_size) + \", Minibatch Loss= \" + \"{:.6f}\".format(loss) + \\\n",
|
||||
" \", Training Accuracy= \" + \"{:.5f}\".format(acc)\n",
|
||||
" step += 1\n",
|
||||
" print \"Optimization Finished!\"\n",
|
||||
" # Calculate accuracy for 256 mnist test images\n",
|
||||
" test_len = 256\n",
|
||||
" test_data = mnist.test.images[:test_len].reshape((-1, n_steps, n_input))\n",
|
||||
" test_label = mnist.test.labels[:test_len]\n",
|
||||
" print \"Testing Accuracy:\", sess.run(accuracy, feed_dict={x: test_data, y: test_label,\n",
|
||||
" istate: np.zeros((test_len, 2*n_hidden))})"
|
||||
]
|
||||
}
|
||||
],
|
||||
"metadata": {
|
||||
"kernelspec": {
|
||||
"display_name": "Python 3",
|
||||
"language": "python",
|
||||
"name": "python3"
|
||||
},
|
||||
"language_info": {
|
||||
"codemirror_mode": {
|
||||
"name": "ipython",
|
||||
"version": 3
|
||||
},
|
||||
"file_extension": ".py",
|
||||
"mimetype": "text/x-python",
|
||||
"name": "python",
|
||||
"nbconvert_exporter": "python",
|
||||
"pygments_lexer": "ipython3",
|
||||
"version": "3.4.3"
|
||||
}
|
||||
},
|
||||
"nbformat": 4,
|
||||
"nbformat_minor": 0
|
||||
}
|
||||
|
|
@ -0,0 +1,184 @@
|
|||
{
|
||||
"cells": [
|
||||
{
|
||||
"cell_type": "markdown",
|
||||
"metadata": {},
|
||||
"source": [
|
||||
"# Basic Multi GPU Computation in TensorFlow\n",
|
||||
"\n",
|
||||
"Credits: Forked from [TensorFlow-Examples](https://github.com/aymericdamien/TensorFlow-Examples) by Aymeric Damien\n",
|
||||
"\n",
|
||||
"## Setup\n",
|
||||
"\n",
|
||||
"Refer to the [setup instructions](http://nbviewer.ipython.org/github/donnemartin/data-science-ipython-notebooks/blob/master/deep-learning/tensor-flow-examples/Setup_TensorFlow.md)"
|
||||
]
|
||||
},
|
||||
{
|
||||
"cell_type": "markdown",
|
||||
"metadata": {
|
||||
"collapsed": true
|
||||
},
|
||||
"source": [
|
||||
"This tutorial requires your machine to have 2 GPUs\n",
|
||||
"* \"/cpu:0\": The CPU of your machine.\n",
|
||||
"* \"/gpu:0\": The first GPU of your machine\n",
|
||||
"* \"/gpu:1\": The second GPU of your machine\n",
|
||||
"* For this example, we are using 2 GTX-980"
|
||||
]
|
||||
},
|
||||
{
|
||||
"cell_type": "code",
|
||||
"execution_count": 2,
|
||||
"metadata": {
|
||||
"collapsed": true
|
||||
},
|
||||
"outputs": [],
|
||||
"source": [
|
||||
"import numpy as np\n",
|
||||
"import tensorflow as tf\n",
|
||||
"import datetime"
|
||||
]
|
||||
},
|
||||
{
|
||||
"cell_type": "code",
|
||||
"execution_count": 3,
|
||||
"metadata": {
|
||||
"collapsed": true
|
||||
},
|
||||
"outputs": [],
|
||||
"source": [
|
||||
"#Processing Units logs\n",
|
||||
"log_device_placement = True\n",
|
||||
"\n",
|
||||
"#num of multiplications to perform\n",
|
||||
"n = 10"
|
||||
]
|
||||
},
|
||||
{
|
||||
"cell_type": "code",
|
||||
"execution_count": null,
|
||||
"metadata": {
|
||||
"collapsed": false
|
||||
},
|
||||
"outputs": [],
|
||||
"source": [
|
||||
"# Example: compute A^n + B^n on 2 GPUs\n",
|
||||
"\n",
|
||||
"# Create random large matrix\n",
|
||||
"A = np.random.rand(1e4, 1e4).astype('float32')\n",
|
||||
"B = np.random.rand(1e4, 1e4).astype('float32')\n",
|
||||
"\n",
|
||||
"# Creates a graph to store results\n",
|
||||
"c1 = []\n",
|
||||
"c2 = []\n",
|
||||
"\n",
|
||||
"# Define matrix power\n",
|
||||
"def matpow(M, n):\n",
|
||||
" if n < 1: #Abstract cases where n < 1\n",
|
||||
" return M\n",
|
||||
" else:\n",
|
||||
" return tf.matmul(M, matpow(M, n-1))"
|
||||
]
|
||||
},
|
||||
{
|
||||
"cell_type": "code",
|
||||
"execution_count": 6,
|
||||
"metadata": {
|
||||
"collapsed": true
|
||||
},
|
||||
"outputs": [],
|
||||
"source": [
|
||||
"# Single GPU computing\n",
|
||||
"\n",
|
||||
"with tf.device('/gpu:0'):\n",
|
||||
" a = tf.constant(A)\n",
|
||||
" b = tf.constant(B)\n",
|
||||
" #compute A^n and B^n and store results in c1\n",
|
||||
" c1.append(matpow(a, n))\n",
|
||||
" c1.append(matpow(b, n))\n",
|
||||
"\n",
|
||||
"with tf.device('/cpu:0'):\n",
|
||||
" sum = tf.add_n(c1) #Addition of all elements in c1, i.e. A^n + B^n\n",
|
||||
"\n",
|
||||
"t1_1 = datetime.datetime.now()\n",
|
||||
"with tf.Session(config=tf.ConfigProto(log_device_placement=log_device_placement)) as sess:\n",
|
||||
" # Runs the op.\n",
|
||||
" sess.run(sum)\n",
|
||||
"t2_1 = datetime.datetime.now()"
|
||||
]
|
||||
},
|
||||
{
|
||||
"cell_type": "code",
|
||||
"execution_count": 7,
|
||||
"metadata": {
|
||||
"collapsed": true
|
||||
},
|
||||
"outputs": [],
|
||||
"source": [
|
||||
"# Multi GPU computing\n",
|
||||
"# GPU:0 computes A^n\n",
|
||||
"with tf.device('/gpu:0'):\n",
|
||||
" #compute A^n and store result in c2\n",
|
||||
" a = tf.constant(A)\n",
|
||||
" c2.append(matpow(a, n))\n",
|
||||
"\n",
|
||||
"#GPU:1 computes B^n\n",
|
||||
"with tf.device('/gpu:1'):\n",
|
||||
" #compute B^n and store result in c2\n",
|
||||
" b = tf.constant(B)\n",
|
||||
" c2.append(matpow(b, n))\n",
|
||||
"\n",
|
||||
"with tf.device('/cpu:0'):\n",
|
||||
" sum = tf.add_n(c2) #Addition of all elements in c2, i.e. A^n + B^n\n",
|
||||
"\n",
|
||||
"t1_2 = datetime.datetime.now()\n",
|
||||
"with tf.Session(config=tf.ConfigProto(log_device_placement=log_device_placement)) as sess:\n",
|
||||
" # Runs the op.\n",
|
||||
" sess.run(sum)\n",
|
||||
"t2_2 = datetime.datetime.now()"
|
||||
]
|
||||
},
|
||||
{
|
||||
"cell_type": "code",
|
||||
"execution_count": 8,
|
||||
"metadata": {
|
||||
"collapsed": false
|
||||
},
|
||||
"outputs": [
|
||||
{
|
||||
"name": "stdout",
|
||||
"output_type": "stream",
|
||||
"text": [
|
||||
"Single GPU computation time: 0:00:11.833497\n",
|
||||
"Multi GPU computation time: 0:00:07.085913\n"
|
||||
]
|
||||
}
|
||||
],
|
||||
"source": [
|
||||
"print \"Single GPU computation time: \" + str(t2_1-t1_1)\n",
|
||||
"print \"Multi GPU computation time: \" + str(t2_2-t1_2)"
|
||||
]
|
||||
}
|
||||
],
|
||||
"metadata": {
|
||||
"kernelspec": {
|
||||
"display_name": "Python 3",
|
||||
"language": "python",
|
||||
"name": "python3"
|
||||
},
|
||||
"language_info": {
|
||||
"codemirror_mode": {
|
||||
"name": "ipython",
|
||||
"version": 3
|
||||
},
|
||||
"file_extension": ".py",
|
||||
"mimetype": "text/x-python",
|
||||
"name": "python",
|
||||
"nbconvert_exporter": "python",
|
||||
"pygments_lexer": "ipython3",
|
||||
"version": "3.4.3"
|
||||
}
|
||||
},
|
||||
"nbformat": 4,
|
||||
"nbformat_minor": 0
|
||||
}
|
||||
File diff suppressed because one or more lines are too long
File diff suppressed because one or more lines are too long
BIN
images/deep.png
Normal file
BIN
images/deep.png
Normal file
Binary file not shown.
|
After Width: | Height: | Size: 27 KiB |
BIN
images/tensorflow.png
Normal file
BIN
images/tensorflow.png
Normal file
Binary file not shown.
|
After Width: | Height: | Size: 8.9 KiB |
BIN
images/theano.png
Normal file
BIN
images/theano.png
Normal file
Binary file not shown.
|
After Width: | Height: | Size: 4.5 KiB |
Loading…
Reference in New Issue
Block a user