StarMirror/data-science-ipython-notebooks
1
0
Fork 0
mirror of https://github.com/donnemartin/data-science-ipython-notebooks.git synced 2024-03-22 13:30:56 +08:00
Code Issues Projects Releases Wiki Activity

Added use cases for S3DistCp.

Browse Source
This commit is contained in:
Donne Martin 2015-02-19 08:48:01 -05:00
parent c8569ed3d5
commit 525da02e17
1 changed files with 4 additions and 2 deletions
Show Stats Download Patch File Download Diff File Expand all files Collapse all files

6
commands/aws.ipynb
View File

@ -1,7 +1,7 @@
{
"metadata": {
"name": "",
"signature": "sha256:f75051e8768199fb18b1e41e2b9e76d496309c12f65bffda915754795767ecb3"
"signature": "sha256:a0e8d0242f04dd2d399759e132a0d586ef06b84f9af2e89105e2835e56d40b2a"
},
"nbformat": 3,
"nbformat_minor": 0,
@ -60,7 +60,9 @@
"cell_type": "markdown",
"metadata": {},
"source": [
"## S3DistCp"
"## S3DistCp\n",
"\n",
"[S3DistCp](http://docs.aws.amazon.com/ElasticMapReduce/latest/DeveloperGuide/UsingEMR_s3distcp.html) is an extension of DistCp that is optimized to work with Amazon S3. S3DistCp is useful for combining smaller files and aggregate them together, taking in a pattern and target file to combine smaller input files to larger ones. S3DistCp can also be used to transfer large volumes of data from S3 to your Hadoop cluster."
]
},
{