Fixed #9: Add notebook info for Boto, the official AWS SDK for Python.

This commit is contained in:
Donne Martin 2015-07-17 08:12:22 -04:00
parent 3ab740904f
commit 49cf71e672
2 changed files with 131 additions and 0 deletions

View File

@ -74,6 +74,7 @@ IPython Notebook(s) demonstrating Amazon Web Services (AWS) and AWS tools functi
| Notebook | Description |
|------------------------------------------------------------------------------------------------------------------------------------|-----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------|
| [boto](http://nbviewer.ipython.org/github/donnemartin/data-science-ipython-notebooks/blob/master/aws/aws.ipynb#boto) | Official AWS SDK for Python. |
| [s3cmd](http://nbviewer.ipython.org/github/donnemartin/data-science-ipython-notebooks/blob/master/aws/aws.ipynb#s3cmd) | Interacts with S3 through the command line. |
| [s3distcp](http://nbviewer.ipython.org/github/donnemartin/data-science-ipython-notebooks/blob/master/aws/aws.ipynb#s3distcp) | Combines smaller files and aggregates them together by taking in a pattern and target file. S3DistCp can also be used to transfer large volumes of data from S3 to your Hadoop cluster. |
| [s3-parallel-put](http://nbviewer.ipython.org/github/donnemartin/data-science-ipython-notebooks/blob/master/aws/aws.ipynb#s3-parallel-put) | Uploads multiple files to S3 in parallel. |

View File

@ -14,6 +14,7 @@
"# Amazon Web Services (AWS)\n",
"\n",
"* SSH to EC2\n",
"* Boto\n",
"* S3cmd\n",
"* s3-parallel-put\n",
"* S3DistCp\n",
@ -65,6 +66,135 @@
"!ssh -i key.pem ec2-user@ipaddress"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"## Boto\n",
"\n",
"[Boto](https://github.com/boto/boto) is the official AWS SDK for Python.\n",
"\n",
"Install Boto:"
]
},
{
"cell_type": "code",
"execution_count": null,
"metadata": {
"collapsed": true
},
"outputs": [],
"source": [
"!pip install Boto"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"Configure boto by creating a ~/.boto file with the following:"
]
},
{
"cell_type": "code",
"execution_count": null,
"metadata": {
"collapsed": true
},
"outputs": [],
"source": [
"aws_access_key_id = YOURACCESSKEY\n",
"aws_secret_access_key = YOURSECRETKEY"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"Work with S3:"
]
},
{
"cell_type": "code",
"execution_count": null,
"metadata": {
"collapsed": true
},
"outputs": [],
"source": [
"import boto\n",
"s3 = boto.connect_s3()"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"Work with EC2:"
]
},
{
"cell_type": "code",
"execution_count": null,
"metadata": {
"collapsed": true
},
"outputs": [],
"source": [
"import boto.ec2\n",
"ec2 = boto.ec2.connect_to_region('us-east-1')"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"Create a bucket and put an object in that bucket:"
]
},
{
"cell_type": "code",
"execution_count": null,
"metadata": {
"collapsed": true
},
"outputs": [],
"source": [
"import boto\n",
"import time\n",
"s3 = boto.connect_s3()\n",
"\n",
"# Create a new bucket. Buckets must have a globally unique name (not just\n",
"# unique to your account).\n",
"bucket = s3.create_bucket('boto-demo-%s' % int(time.time()))\n",
"\n",
"# Create a new key/value pair.\n",
"key = bucket.new_key('mykey')\n",
"key.set_contents_from_string(\"Hello World!\")\n",
"\n",
"# Sleep to ensure the data is eventually there.\n",
"# This is often referred to as \"S3 eventual consistency\".\n",
"time.sleep(2)\n",
"\n",
"# Retrieve the contents of ``mykey``.\n",
"print key.get_contents_as_string()\n",
"\n",
"# Delete the key.\n",
"key.delete()\n",
"\n",
"# Delete the bucket.\n",
"bucket.delete()"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"Each service supports a different set of commands. Refer to the following for more details:\n",
"* [AWS Docs](https://aws.amazon.com/documentation/)\n",
"* [Boto Docs](http://boto.readthedocs.org/en/latest/index.html)"
]
},
{
"cell_type": "markdown",
"metadata": {},