mirror of
https://github.com/donnemartin/data-science-ipython-notebooks.git
synced 2024-03-22 13:30:56 +08:00
Add new pandas notebooks
Source: https://github.com/jakevdp/PythonDataScienceHandbook unmodified
This commit is contained in:
parent
d52331cd5a
commit
233b197660
164
pandas/03.00-Introduction-to-Pandas.ipynb
Normal file
164
pandas/03.00-Introduction-to-Pandas.ipynb
Normal file
|
@ -0,0 +1,164 @@
|
||||||
|
{
|
||||||
|
"cells": [
|
||||||
|
{
|
||||||
|
"cell_type": "markdown",
|
||||||
|
"metadata": {},
|
||||||
|
"source": [
|
||||||
|
"<!--BOOK_INFORMATION-->\n",
|
||||||
|
"<img align=\"left\" style=\"padding-right:10px;\" src=\"figures/PDSH-cover-small.png\">\n",
|
||||||
|
"*This notebook contains an excerpt from the [Python Data Science Handbook](http://shop.oreilly.com/product/0636920034919.do) by Jake VanderPlas; the content is available [on GitHub](https://github.com/jakevdp/PythonDataScienceHandbook).*\n",
|
||||||
|
"\n",
|
||||||
|
"*The text is released under the [CC-BY-NC-ND license](https://creativecommons.org/licenses/by-nc-nd/3.0/us/legalcode), and code is released under the [MIT license](https://opensource.org/licenses/MIT). If you find this content useful, please consider supporting the work by [buying the book](http://shop.oreilly.com/product/0636920034919.do)!*\n",
|
||||||
|
"\n",
|
||||||
|
"*No changes were made to the contents of this notebook from the original.*"
|
||||||
|
]
|
||||||
|
},
|
||||||
|
{
|
||||||
|
"cell_type": "markdown",
|
||||||
|
"metadata": {},
|
||||||
|
"source": [
|
||||||
|
"<!--NAVIGATION-->\n",
|
||||||
|
"< [Structured Data: NumPy's Structured Arrays](02.09-Structured-Data-NumPy.ipynb) | [Contents](Index.ipynb) | [Introducing Pandas Objects](03.01-Introducing-Pandas-Objects.ipynb) >"
|
||||||
|
]
|
||||||
|
},
|
||||||
|
{
|
||||||
|
"cell_type": "markdown",
|
||||||
|
"metadata": {},
|
||||||
|
"source": [
|
||||||
|
"# Data Manipulation with Pandas"
|
||||||
|
]
|
||||||
|
},
|
||||||
|
{
|
||||||
|
"cell_type": "markdown",
|
||||||
|
"metadata": {},
|
||||||
|
"source": [
|
||||||
|
"In the previous chapter, we dove into detail on NumPy and its ``ndarray`` object, which provides efficient storage and manipulation of dense typed arrays in Python.\n",
|
||||||
|
"Here we'll build on this knowledge by looking in detail at the data structures provided by the Pandas library.\n",
|
||||||
|
"Pandas is a newer package built on top of NumPy, and provides an efficient implementation of a ``DataFrame``.\n",
|
||||||
|
"``DataFrame``s are essentially multidimensional arrays with attached row and column labels, and often with heterogeneous types and/or missing data.\n",
|
||||||
|
"As well as offering a convenient storage interface for labeled data, Pandas implements a number of powerful data operations familiar to users of both database frameworks and spreadsheet programs.\n",
|
||||||
|
"\n",
|
||||||
|
"As we saw, NumPy's ``ndarray`` data structure provides essential features for the type of clean, well-organized data typically seen in numerical computing tasks.\n",
|
||||||
|
"While it serves this purpose very well, its limitations become clear when we need more flexibility (e.g., attaching labels to data, working with missing data, etc.) and when attempting operations that do not map well to element-wise broadcasting (e.g., groupings, pivots, etc.), each of which is an important piece of analyzing the less structured data available in many forms in the world around us.\n",
|
||||||
|
"Pandas, and in particular its ``Series`` and ``DataFrame`` objects, builds on the NumPy array structure and provides efficient access to these sorts of \"data munging\" tasks that occupy much of a data scientist's time.\n",
|
||||||
|
"\n",
|
||||||
|
"In this chapter, we will focus on the mechanics of using ``Series``, ``DataFrame``, and related structures effectively.\n",
|
||||||
|
"We will use examples drawn from real datasets where appropriate, but these examples are not necessarily the focus."
|
||||||
|
]
|
||||||
|
},
|
||||||
|
{
|
||||||
|
"cell_type": "markdown",
|
||||||
|
"metadata": {},
|
||||||
|
"source": [
|
||||||
|
"## Installing and Using Pandas\n",
|
||||||
|
"\n",
|
||||||
|
"Installation of Pandas on your system requires NumPy to be installed, and if building the library from source, requires the appropriate tools to compile the C and Cython sources on which Pandas is built.\n",
|
||||||
|
"Details on this installation can be found in the [Pandas documentation](http://pandas.pydata.org/).\n",
|
||||||
|
"If you followed the advice outlined in the [Preface](00.00-Preface.ipynb) and used the Anaconda stack, you already have Pandas installed.\n",
|
||||||
|
"\n",
|
||||||
|
"Once Pandas is installed, you can import it and check the version:"
|
||||||
|
]
|
||||||
|
},
|
||||||
|
{
|
||||||
|
"cell_type": "code",
|
||||||
|
"execution_count": 1,
|
||||||
|
"metadata": {
|
||||||
|
"collapsed": false
|
||||||
|
},
|
||||||
|
"outputs": [
|
||||||
|
{
|
||||||
|
"data": {
|
||||||
|
"text/plain": [
|
||||||
|
"'0.18.1'"
|
||||||
|
]
|
||||||
|
},
|
||||||
|
"execution_count": 1,
|
||||||
|
"metadata": {},
|
||||||
|
"output_type": "execute_result"
|
||||||
|
}
|
||||||
|
],
|
||||||
|
"source": [
|
||||||
|
"import pandas\n",
|
||||||
|
"pandas.__version__"
|
||||||
|
]
|
||||||
|
},
|
||||||
|
{
|
||||||
|
"cell_type": "markdown",
|
||||||
|
"metadata": {},
|
||||||
|
"source": [
|
||||||
|
"Just as we generally import NumPy under the alias ``np``, we will import Pandas under the alias ``pd``:"
|
||||||
|
]
|
||||||
|
},
|
||||||
|
{
|
||||||
|
"cell_type": "code",
|
||||||
|
"execution_count": 2,
|
||||||
|
"metadata": {
|
||||||
|
"collapsed": true
|
||||||
|
},
|
||||||
|
"outputs": [],
|
||||||
|
"source": [
|
||||||
|
"import pandas as pd"
|
||||||
|
]
|
||||||
|
},
|
||||||
|
{
|
||||||
|
"cell_type": "markdown",
|
||||||
|
"metadata": {},
|
||||||
|
"source": [
|
||||||
|
"This import convention will be used throughout the remainder of this book."
|
||||||
|
]
|
||||||
|
},
|
||||||
|
{
|
||||||
|
"cell_type": "markdown",
|
||||||
|
"metadata": {},
|
||||||
|
"source": [
|
||||||
|
"## Reminder about Built-In Documentation\n",
|
||||||
|
"\n",
|
||||||
|
"As you read through this chapter, don't forget that IPython gives you the ability to quickly explore the contents of a package (by using the tab-completion feature) as well as the documentation of various functions (using the ``?`` character). (Refer back to [Help and Documentation in IPython](01.01-Help-And-Documentation.ipynb) if you need a refresher on this.)\n",
|
||||||
|
"\n",
|
||||||
|
"For example, to display all the contents of the pandas namespace, you can type\n",
|
||||||
|
"\n",
|
||||||
|
"```ipython\n",
|
||||||
|
"In [3]: pd.<TAB>\n",
|
||||||
|
"```\n",
|
||||||
|
"\n",
|
||||||
|
"And to display Pandas's built-in documentation, you can use this:\n",
|
||||||
|
"\n",
|
||||||
|
"```ipython\n",
|
||||||
|
"In [4]: pd?\n",
|
||||||
|
"```\n",
|
||||||
|
"\n",
|
||||||
|
"More detailed documentation, along with tutorials and other resources, can be found at http://pandas.pydata.org/."
|
||||||
|
]
|
||||||
|
},
|
||||||
|
{
|
||||||
|
"cell_type": "markdown",
|
||||||
|
"metadata": {},
|
||||||
|
"source": [
|
||||||
|
"<!--NAVIGATION-->\n",
|
||||||
|
"< [Structured Data: NumPy's Structured Arrays](02.09-Structured-Data-NumPy.ipynb) | [Contents](Index.ipynb) | [Introducing Pandas Objects](03.01-Introducing-Pandas-Objects.ipynb) >"
|
||||||
|
]
|
||||||
|
}
|
||||||
|
],
|
||||||
|
"metadata": {
|
||||||
|
"anaconda-cloud": {},
|
||||||
|
"kernelspec": {
|
||||||
|
"display_name": "Python 3",
|
||||||
|
"language": "python",
|
||||||
|
"name": "python3"
|
||||||
|
},
|
||||||
|
"language_info": {
|
||||||
|
"codemirror_mode": {
|
||||||
|
"name": "ipython",
|
||||||
|
"version": 3
|
||||||
|
},
|
||||||
|
"file_extension": ".py",
|
||||||
|
"mimetype": "text/x-python",
|
||||||
|
"name": "python",
|
||||||
|
"nbconvert_exporter": "python",
|
||||||
|
"pygments_lexer": "ipython3",
|
||||||
|
"version": "3.4.3"
|
||||||
|
}
|
||||||
|
},
|
||||||
|
"nbformat": 4,
|
||||||
|
"nbformat_minor": 0
|
||||||
|
}
|
1560
pandas/03.01-Introducing-Pandas-Objects.ipynb
Normal file
1560
pandas/03.01-Introducing-Pandas-Objects.ipynb
Normal file
File diff suppressed because it is too large
Load Diff
1601
pandas/03.02-Data-Indexing-and-Selection.ipynb
Normal file
1601
pandas/03.02-Data-Indexing-and-Selection.ipynb
Normal file
File diff suppressed because it is too large
Load Diff
1035
pandas/03.03-Operations-in-Pandas.ipynb
Normal file
1035
pandas/03.03-Operations-in-Pandas.ipynb
Normal file
File diff suppressed because it is too large
Load Diff
1296
pandas/03.04-Missing-Values.ipynb
Normal file
1296
pandas/03.04-Missing-Values.ipynb
Normal file
File diff suppressed because it is too large
Load Diff
2520
pandas/03.05-Hierarchical-Indexing.ipynb
Normal file
2520
pandas/03.05-Hierarchical-Indexing.ipynb
Normal file
File diff suppressed because it is too large
Load Diff
1637
pandas/03.06-Concat-And-Append.ipynb
Normal file
1637
pandas/03.06-Concat-And-Append.ipynb
Normal file
File diff suppressed because it is too large
Load Diff
3570
pandas/03.07-Merge-and-Join.ipynb
Normal file
3570
pandas/03.07-Merge-and-Join.ipynb
Normal file
File diff suppressed because it is too large
Load Diff
2657
pandas/03.08-Aggregation-and-Grouping.ipynb
Normal file
2657
pandas/03.08-Aggregation-and-Grouping.ipynb
Normal file
File diff suppressed because it is too large
Load Diff
1376
pandas/03.09-Pivot-Tables.ipynb
Normal file
1376
pandas/03.09-Pivot-Tables.ipynb
Normal file
File diff suppressed because one or more lines are too long
1404
pandas/03.10-Working-With-Strings.ipynb
Normal file
1404
pandas/03.10-Working-With-Strings.ipynb
Normal file
File diff suppressed because it is too large
Load Diff
1957
pandas/03.11-Working-with-Time-Series.ipynb
Normal file
1957
pandas/03.11-Working-with-Time-Series.ipynb
Normal file
File diff suppressed because one or more lines are too long
1147
pandas/03.12-Performance-Eval-and-Query.ipynb
Normal file
1147
pandas/03.12-Performance-Eval-and-Query.ipynb
Normal file
File diff suppressed because it is too large
Load Diff
76
pandas/03.13-Further-Resources.ipynb
Normal file
76
pandas/03.13-Further-Resources.ipynb
Normal file
|
@ -0,0 +1,76 @@
|
||||||
|
{
|
||||||
|
"cells": [
|
||||||
|
{
|
||||||
|
"cell_type": "markdown",
|
||||||
|
"metadata": {},
|
||||||
|
"source": [
|
||||||
|
"<!--BOOK_INFORMATION-->\n",
|
||||||
|
"<img align=\"left\" style=\"padding-right:10px;\" src=\"figures/PDSH-cover-small.png\">\n",
|
||||||
|
"*This notebook contains an excerpt from the [Python Data Science Handbook](http://shop.oreilly.com/product/0636920034919.do) by Jake VanderPlas; the content is available [on GitHub](https://github.com/jakevdp/PythonDataScienceHandbook).*\n",
|
||||||
|
"\n",
|
||||||
|
"*The text is released under the [CC-BY-NC-ND license](https://creativecommons.org/licenses/by-nc-nd/3.0/us/legalcode), and code is released under the [MIT license](https://opensource.org/licenses/MIT). If you find this content useful, please consider supporting the work by [buying the book](http://shop.oreilly.com/product/0636920034919.do)!*\n",
|
||||||
|
"\n",
|
||||||
|
"*No changes were made to the contents of this notebook from the original.*"
|
||||||
|
]
|
||||||
|
},
|
||||||
|
{
|
||||||
|
"cell_type": "markdown",
|
||||||
|
"metadata": {},
|
||||||
|
"source": [
|
||||||
|
"<!--NAVIGATION-->\n",
|
||||||
|
"< [High-Performance Pandas: eval() and query()](03.12-Performance-Eval-and-Query.ipynb) | [Contents](Index.ipynb) | [Visualization with Matplotlib](04.00-Introduction-To-Matplotlib.ipynb) >"
|
||||||
|
]
|
||||||
|
},
|
||||||
|
{
|
||||||
|
"cell_type": "markdown",
|
||||||
|
"metadata": {},
|
||||||
|
"source": [
|
||||||
|
"# Further Resources\n",
|
||||||
|
"\n",
|
||||||
|
"In this chapter, we've covered many of the basics of using Pandas effectively for data analysis.\n",
|
||||||
|
"Still, much has been omitted from our discussion.\n",
|
||||||
|
"To learn more about Pandas, I recommend the following resources:\n",
|
||||||
|
"\n",
|
||||||
|
"- [Pandas online documentation](http://pandas.pydata.org/): This is the go-to source for complete documentation of the package. While the examples in the documentation tend to be small generated datasets, the description of the options is complete and generally very useful for understanding the use of various functions.\n",
|
||||||
|
"\n",
|
||||||
|
"- [*Python for Data Analysis*](http://shop.oreilly.com/product/0636920023784.do) Written by Wes McKinney (the original creator of Pandas), this book contains much more detail on the Pandas package than we had room for in this chapter. In particular, he takes a deep dive into tools for time series, which were his bread and butter as a financial consultant. The book also has many entertaining examples of applying Pandas to gain insight from real-world datasets. Keep in mind, though, that the book is now several years old, and the Pandas package has quite a few new features that this book does not cover (but be on the lookout for a new edition in 2017).\n",
|
||||||
|
"\n",
|
||||||
|
"- [Stack Overflow](http://stackoverflow.com/questions/tagged/pandas): Pandas has so many users that any question you have has likely been asked and answered on Stack Overflow. Using Pandas is a case where some Google-Fu is your best friend. Simply go to your favorite search engine and type in the question, problem, or error you're coming across–more than likely you'll find your answer on a Stack Overflow page.\n",
|
||||||
|
"\n",
|
||||||
|
"- [Pandas on PyVideo](http://pyvideo.org/search?q=pandas): From PyCon to SciPy to PyData, many conferences have featured tutorials from Pandas developers and power users. The PyCon tutorials in particular tend to be given by very well-vetted presenters.\n",
|
||||||
|
"\n",
|
||||||
|
"Using these resources, combined with the walk-through given in this chapter, my hope is that you'll be poised to use Pandas to tackle any data analysis problem you come across!"
|
||||||
|
]
|
||||||
|
},
|
||||||
|
{
|
||||||
|
"cell_type": "markdown",
|
||||||
|
"metadata": {},
|
||||||
|
"source": [
|
||||||
|
"<!--NAVIGATION-->\n",
|
||||||
|
"< [High-Performance Pandas: eval() and query()](03.12-Performance-Eval-and-Query.ipynb) | [Contents](Index.ipynb) | [Visualization with Matplotlib](04.00-Introduction-To-Matplotlib.ipynb) >"
|
||||||
|
]
|
||||||
|
}
|
||||||
|
],
|
||||||
|
"metadata": {
|
||||||
|
"anaconda-cloud": {},
|
||||||
|
"kernelspec": {
|
||||||
|
"display_name": "Python 3",
|
||||||
|
"language": "python",
|
||||||
|
"name": "python3"
|
||||||
|
},
|
||||||
|
"language_info": {
|
||||||
|
"codemirror_mode": {
|
||||||
|
"name": "ipython",
|
||||||
|
"version": 3
|
||||||
|
},
|
||||||
|
"file_extension": ".py",
|
||||||
|
"mimetype": "text/x-python",
|
||||||
|
"name": "python",
|
||||||
|
"nbconvert_exporter": "python",
|
||||||
|
"pygments_lexer": "ipython3",
|
||||||
|
"version": "3.4.3"
|
||||||
|
}
|
||||||
|
},
|
||||||
|
"nbformat": 4,
|
||||||
|
"nbformat_minor": 0
|
||||||
|
}
|
Loading…
Reference in New Issue
Block a user