data-science-ipython-notebooks/commands/misc.ipynb

{
 "cells": [
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "This notebook was prepared by [Donne Martin](http://donnemartin.com). Source and license info is on [GitHub](https://github.com/donnemartin/data-science-ipython-notebooks)."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "# Misc Commands\n",
    "\n",
    "* Anaconda\n",
    "* IPython Notebook\n",
    "* Git\n",
    "* Ruby\n",
    "* Jekyll\n",
    "* Pelican\n",
    "* Django"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "<h2 id=\"anaconda\">Anaconda</h2>"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "[Anaconda](https://store.continuum.io/cshop/anaconda/) is a scientific python distribution containing Python, NumPy, SciPy, Pandas, IPython, Matplotlib, Numba, Blaze, Bokeh, and other great Python data analysis tools."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "collapsed": false
   },
   "outputs": [],
   "source": [
    "# See Anaconda installed packages\n",
    "!conda list\n",
    "\n",
    "# List environments\n",
    "!conda info -e\n",
    "\n",
    "# Create Python 3 environment\n",
    "!conda create -n py3k python=3 anaconda\n",
    "\n",
    "# Activate Python 3 environment\n",
    "!source activate py3k\n",
    "\n",
    "# Deactivate Python 3 environment\n",
    "!source deactivate\n",
    "\n",
    "# Update Anaconda\n",
    "!conda update conda\n",
    "\n",
    "# Update a package with Anaconda\n",
    "!conda update ipython\n",
    "\n",
    "# Update a package\n",
    "!conda update scipy\n",
    "\n",
    "# Update all packages\n",
    "!conda update all\n",
    "\n",
    "# Install specific version of a package\n",
    "!conda install scipy=0.12.0\n",
    "\n",
    "# Cleanup: Conda can accumulate a lot of disk space\n",
    "# because it doesn’t remove old unused packages\n",
    "!conda clean -p\n",
    "\n",
    "# Cleanup tarballs which are kept for caching purposes\n",
    "!conda clean -t"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "<h2 id=\"ipython-notebook\">IPython Notebook</h2>"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "[IPython Notebook](http://ipython.org/notebook.html) is a \"web-based interactive computational environment where you can combine code execution, text, mathematics, plots and rich media into a single document.\""
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "collapsed": false
   },
   "outputs": [],
   "source": [
    "# Start IPython Notebook\n",
    "ipython notebook\n",
    "\n",
    "# Start IPython Notebook with built-in mode to work cleanly \n",
    "# with matplotlib figures\n",
    "ipython notebook --pylab inline\n",
    "\n",
    "# Start IPython Notebook with a profile\n",
    "ipython notebook --profile=dark-bg\n",
    "\n",
    "# Load the contents of a file\n",
    "%load dir/file.py\n",
    "\n",
    "# Time execution of a Python statement or expression\n",
    "%timeit\n",
    "%%time\n",
    "\n",
    "# Activate the interactive debugger\n",
    "%debug\n",
    "\n",
    "# Write the contents of the cell to a file\n",
    "%writefile\n",
    "\n",
    "# Run a cell via a shell command\n",
    "%%script\n",
    "\n",
    "# Run cells with bash in a subprocess\n",
    "# This is a shortcut for %%script bash\n",
    "%%bash\n",
    "\n",
    "# Run cells with python2 in a subprocess\n",
    "%%python2\n",
    "\n",
    "# Run cells with python3 in a subprocess\n",
    "%%python3\n",
    "\n",
    "# Convert a notebook to a basic HTML file \n",
    "!ipython nbconvert --to html --template basic file.ipynb "
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "| Command   | Description                              |\n",
    "|-----------|------------------------------------------|\n",
    "| ?         | Intro and overview of IPython's features |\n",
    "| %quickref | Quick reference                          |\n",
    "| help      | Python help                              |\n",
    "| object?   | Object details, also use object??        |"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Apply css styling based on a css file:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "collapsed": true
   },
   "outputs": [],
   "source": [
    "from IPython.core.display import HTML\n",
    "\n",
    "def css_styling():\n",
    "    styles = open(\"styles/custom.css\", \"r\").read()\n",
    "    return HTML(styles)\n",
    "css_styling()"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "<h2 id=\"git\">Git</h2>"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "[Git](http://git-scm.com/) is a distributed revision control system."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "collapsed": false
   },
   "outputs": [],
   "source": [
    "# Configure git\n",
    "!git config --global user.name 'First Last'\n",
    "!git config --global user.email 'name@domain.com'\n",
    "!git init\n",
    "\n",
    "# View status and log\n",
    "!git status\n",
    "!git log\n",
    "\n",
    "# Add or remove from staging area\n",
    "!git add [target]\n",
    "!git reset [target file or commit]\n",
    "!git reset --hard origin/master\n",
    "\n",
    "# Automatically stage tracked files, \n",
    "# including deleting the previously tracked files\n",
    "# Does not add untracked files\n",
    "!git add -u\n",
    "\n",
    "# Delete files and stage them\n",
    "!git rm [target]\n",
    "\n",
    "# Commit\n",
    "!git commit -m “Add commit message here”\n",
    "\n",
    "# Add new origin\n",
    "!git remote add origin https://github.com/donnemartin/ipython-data-notebooks.git\n",
    "\n",
    "# Set to new origin\n",
    "!git remote set-url origin https://github.com/donnemartin/pydatasnippets.git\n",
    "    \n",
    "# Push to master, -u saves config so you can just do \"git push\" afterwards\n",
    "!git push -u origin master\n",
    "!git push\n",
    "\n",
    "# Diff files\n",
    "!git diff HEAD\n",
    "!git diff --staged\n",
    "!git diff --cached\n",
    "\n",
    "# Show log message of commit and diff\n",
    "!git show $COMMIT\n",
    "\n",
    "# Undo a file that has not been added\n",
    "!git checkout — [target]\n",
    "\n",
    "# Revert a commit\n",
    "!git revert\n",
    "\n",
    "# Undo a push and leave local repo intact\n",
    "!git push -f origin HEAD^:master\n",
    "\n",
    "# Undo commit but leave files and index\n",
    "!git reset --soft HEAD~1\n",
    "\n",
    "# Amend commit message of most recent change\n",
    "!git commit --amend\n",
    "!git push --force [branch]\n",
    "\n",
    "# Take the dirty state of your working directory\n",
    "# and save it on a stack of unfinished changes\n",
    "!git stash\n",
    "\n",
    "# Get list of stashes\n",
    "!git stash list\n",
    "\n",
    "# Apply the top stash, re-modifying the \n",
    "# uncommitted files when the stash was saved\n",
    "!git stash apply\n",
    "\n",
    "# Apply a stash at the specified index\n",
    "!git stash apply stash@{1}\n",
    "\n",
    "# Create a branch\n",
    "!git branch [branch]\n",
    "\n",
    "# Check branches\n",
    "!git branch\n",
    "\n",
    "# Switch branches\n",
    "!git checkout [branch]\n",
    "\n",
    "# Merge branch to master\n",
    "!git merge [branch]\n",
    "\n",
    "# Delete branch\n",
    "!git branch -d [branch]\n",
    "\n",
    "# Clone\n",
    "!git clone git@github.com:repo folder-name\n",
    "!git clone https://donnemartin@bitbucket.org/donnemartin/tutorial.git\n",
    "    \n",
    "# Update a local repository with changes from a remote repository\n",
    "# (pull down from master)\n",
    "!git pull origin master\n",
    "\n",
    "# Configuring a remote for a fork\n",
    "!git remote add upstream [target]\n",
    "\n",
    "# Set remote upstream\n",
    "git branch --set-upstream-to origin/branch\n",
    "\n",
    "# Check remotes\n",
    "!git remote -v\n",
    "\n",
    "# Syncing a fork\n",
    "!git fetch upstream\n",
    "!git checkout master\n",
    "!git merge upstream/master\n",
    "\n",
    "# Create a file containing a patch\n",
    "# git format-patch are like normal patch files, but they also carry information \n",
    "# about the git commit that created the patch: the author, the date, and the \n",
    "# commit log message are all there at the top of the patch.\n",
    "!git format-patch origin/master\n",
    "\n",
    "# Clean up .git folder:\n",
    "!git repack -a -d --depth=250 --window=250\n",
    "\n",
    "# GitHub tutorial:\n",
    "http://try.github.io/levels/1/challenges/9\n",
    "\n",
    "# BitBucket Setup\n",
    "!cd /path/to/my/repo\n",
    "!git init\n",
    "!git remote add origin https://donnemartin@bitbucket.org/donnemartin/repo.git\n",
    "!git push -u origin --all # pushes up the repo and its refs for the first time\n",
    "!git push -u origin --tags # pushes up any tags\n",
    "\n",
    "# Open Hatch missions\n",
    "!git clone https://openhatch.org/git-mission-data/git/dmartin git_missions"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "<h2 id=\"ruby\">Ruby</h2>"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "[Ruby](https://www.ruby-lang.org/en/) is used to interact with the AWS command line and for Jekyll, a blog framework that can be hosted on GitHub Pages."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "collapsed": false
   },
   "outputs": [],
   "source": [
    "# Update Ruby\n",
    "!rvm get stable\n",
    "\n",
    "# Reload Ruby (or open a new terminal)\n",
    "!rvm reload\n",
    "\n",
    "# List all known RVM installable rubies\n",
    "!rvm list known\n",
    "\n",
    "# List all installed Ruby versions\n",
    "!rvm list\n",
    "\n",
    "# Install a specific Ruby version\n",
    "!rvm install 2.1.5\n",
    "\n",
    "# Set Ruby version\n",
    "!rvm --default ruby-1.8.7\n",
    "!rvm --default ruby-2.1.5\n",
    "\n",
    "# Check Ruby version\n",
    "!ruby -v"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "<h2 id=\"jekyll\">Jekyll</h2>"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "[Jekyll](http://jekyllrb.com/) is a blog framework that can be hosted on GitHub Pages.\n",
    "\n",
    "In addition to donnemartin.com, I’ve started to build up its mirror site donnemartin.github.io to try out Jekyll. So far I love that I can use my existing developer tools to generate content (SublimeText, Terminal, and GitHub).\n",
    "\n",
    "Here are other features I like about Jekyll:\n",
    "\n",
    "* Converts Markdown to produce fast, static pages\n",
    "* Simple to get started, no backend or manual updates\n",
    "* Hosted on GitHub Pages\n",
    "* Open source on GitHub"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Many Jekyll themes require a Ruby version of 2 and above.  However, the AWS CLI requires Ruby 1.8.7.  Run the proper version of Ruby for Jekyll:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "collapsed": false
   },
   "outputs": [],
   "source": [
    "!rvm --default ruby-2.1.5"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Build and run the localy Jekyll server:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "collapsed": false
   },
   "outputs": [],
   "source": [
    "# => The current folder will be generated into ./_site\n",
    "!bundle exec jekyll build\n",
    "\n",
    "# => A development server will run at http://localhost:4000/\n",
    "# Auto-regeneration: enabled. Use `--no-watch` to disable.\n",
    "!bundle exec jekyll serve"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "<h2 id=\"pelican\">Pelican</h2>"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "I've switched my personal website [donnemartin.com](http://donnemartin.com/) to run off Pelican, a python-based alternative to Jekyll.  Previous iterations ran off Wordpress and Jekyll.\n",
    "\n",
    "Setup [reference](http://nafiulis.me/making-a-static-blog-with-pelican.html)."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "collapsed": true
   },
   "outputs": [],
   "source": [
    "# Install\n",
    "!pip install pelican\n",
    "!pip install markdown\n",
    "!pip install ghp-import\n",
    "\n",
    "# Quick retup\n",
    "!pelican-quickstart\n",
    "\n",
    "# Run server\n",
    "!make devserver\n",
    "\n",
    "# Stop server\n",
    "!make stopserver\n",
    "\n",
    "# Run ghp-import on output folder\n",
    "# Review https://pypi.python.org/pypi/ghp-import\n",
    "# There's a \"Big Fat Warning\" section\n",
    "!ghp-import output\n",
    "\n",
    "# Update gh-pages (if using a project page)\n",
    "!git push origin gh-pages\n",
    "\n",
    "# Update gh-pages (if using a user or org page)\n",
    "!git merge gh-pages master"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Django"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "[Django](https://www.djangoproject.com) is a high-level Python Web framework that encourages rapid development and clean, pragmatic design.  It can be useful to share reports/analyses and for blogging. Lighter-weight alternatives include [Pyramid](https://github.com/Pylons/pyramid), [Flask](https://github.com/mitsuhiko/flask), [Tornado](https://github.com/tornadoweb/tornado), and [Bottle](https://github.com/bottlepy/bottle)."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "collapsed": true
   },
   "outputs": [],
   "source": [
    "# Check version of Django\n",
    "!python -c \"import django; print(django.get_version())\"\n",
    "\n",
    "# Create and setup a project\n",
    "!django-admin startproject mysite\n",
    "\n",
    "# Sync db\n",
    "!python manage.py syncdb\n",
    "\n",
    "# The migrate command looks at the INSTALLED_APPS setting and \n",
    "# creates any necessary database tables according to the database \n",
    "# settings in your mysite/settings.py file and the database \n",
    "# migrations shipped with the app\n",
    "!python manage.py migrate\n",
    "\n",
    "# Run the dev server\n",
    "!python manage.py runserver\n",
    "1python manage.py runserver 8080\n",
    "!python manage.py runserver 0.0.0.0:8000\n",
    "\n",
    "# Create app\n",
    "!python manage.py startapp [app_label]\n",
    "\n",
    "# Run tests\n",
    "python manage.py test [app_label]\n",
    "\n",
    "# Tell Django that you’ve made some changes to your models \n",
    "# and that you’d like the changes to be stored as a migration.\n",
    "!python manage.py makemigrations [app_label]\n",
    "\n",
    "# Take migration names and returns their SQL\n",
    "!python manage.py sqlmigrate [app_label] [migration_number]\n",
    "\n",
    "# Checks for any problems in your project without making \n",
    "# migrations or touching the database.\n",
    "!python manage.py check\n",
    "\n",
    "# Create a user who can login to the admin site\n",
    "!python manage.py createsuperuser\n",
    "\n",
    "# Locate Django source files\n",
    "!python -c \"\n",
    "import sys\n",
    "sys.path = sys.path[1:]\n",
    "import django\n",
    "print(django.__path__)\""
   ]
  }
 ],
 "metadata": {
  "kernelspec": {
   "display_name": "Python 3",
   "language": "python",
   "name": "python3"
  },
  "language_info": {
   "codemirror_mode": {
    "name": "ipython",
    "version": 3
   },
   "file_extension": ".py",
   "mimetype": "text/x-python",
   "name": "python",
   "nbconvert_exporter": "python",
   "pygments_lexer": "ipython3",
   "version": "3.4.3"
  }
 },
 "nbformat": 4,
 "nbformat_minor": 0
}