diff --git a/README.md b/README.md index 7e597d6..0671707 100644 --- a/README.md +++ b/README.md @@ -14,12 +14,19 @@ IPython Notebooks used in [kaggle](https://www.kaggle.com/) competitions. IPython Notebooks demonstrating Amazon Web Services functionality. * [aws commands index](http://nbviewer.ipython.org/github/donnemartin/ipython-data-notebooks/blob/master/aws/aws.ipynb) + * [s3cmd](http://nbviewer.ipython.org/github/donnemartin/ipython-data-notebooks/blob/master/aws/aws.ipynb#s3cmd): Interacts with S3 through the command line. + * [s3-parallel-put](http://nbviewer.ipython.org/github/donnemartin/ipython-data-notebooks/blob/master/aws/aws.ipynb#s3-parallel-put): Uploads multiple files to S3 in parallel. + * [s3distcp](http://nbviewer.ipython.org/github/donnemartin/ipython-data-notebooks/blob/master/aws/aws.ipynb#s3distcp): Combines smaller files and aggregates them together by taking in a pattern and target file. S3DistCp can also be used to transfer large volumes of data from S3 to your Hadoop cluster. + * [mrjob](http://nbviewer.ipython.org/github/donnemartin/ipython-data-notebooks/blob/master/aws/aws.ipynb#mrjob): Supports MapReduce jobs in Python 2.5+ and runs them locally or on Hadoop clusters. + * [redshift](http://nbviewer.ipython.org/github/donnemartin/ipython-data-notebooks/blob/master/aws/aws.ipynb#redshift): Acts as a fast data warehouse built on top of technology from massive parallel processing (MPP). + * [kinesis](http://nbviewer.ipython.org/github/donnemartin/ipython-data-notebooks/blob/master/aws/aws.ipynb#kinesis): Streams data in real time with the ability to process thousands of data streams per second. + * [lambda](http://nbviewer.ipython.org/github/donnemartin/ipython-data-notebooks/blob/master/aws/aws.ipynb#lambda): Runs code in response to events, automatically managing compute resources. ## spark @@ -27,6 +34,7 @@ IPython Notebooks demonstrating Amazon Web Services functionality. IPython Notebooks demonstrating spark and HDFS functionality. * [spark](http://nbviewer.ipython.org/github/donnemartin/ipython-data-notebooks/blob/master/spark/spark.ipynb): Open-source in-memory cluster computing framework, up to 100 times faster for certain applications and is well suited for machine learning algorithms. + * [hdfs](http://nbviewer.ipython.org/github/donnemartin/ipython-data-notebooks/blob/master/spark/hdfs.ipynb): Reliably stores very large files across machines in a large cluster. ## python-core