{ "cells": [ { "cell_type": "markdown", "metadata": {}, "source": [ "# Tutorial de Panda" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "Pandas es una librería que proporciona estructuras de datos flexibles y permite trabajar con la información de forma eficiente" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "http://pandas.pydata.org" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "Pandas ofrece varias estructuras de datos que nos resultarán de mucha utilidad. Todas las posibles estructuras son:\n", "\n", " Series (y TimeSeries), DataFrame, Panel, Panel4D, PanelND" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "### Series" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "En una instancia de la clase Series podremos almacenar arrays o vectores con índice o etiqueta, incluso funciones. Si no usamos índice o etiqueta nos lo numerará con un índice de forma interna. La forma básica de crear una Series sería:" ] }, { "cell_type": "code", "execution_count": 13, "metadata": { "collapsed": false }, "outputs": [ { "name": "stdout", "output_type": "stream", "text": [ "Serie con índices automáticos\n", "0 1.831633\n", "1 1.003166\n", "2 -1.884045\n", "3 -0.405001\n", "4 1.683828\n", "5 -0.157700\n", "6 -0.525094\n", "7 -0.640867\n", "8 1.138279\n", "9 0.211142\n", "dtype: float64\n", "\n" ] } ], "source": [ "import numpy as np #Importamos la librería numpy\n", "import pandas as pd #Importamos la librería pandas\n", "\n", "\n", "# Ejemplo de serie con índices automáticos\n", "serie = pd.Series(np.random.randn(10))\n", "print(u'Serie con índices automáticos')\n", "print(format(serie))\n", "print(type(serie))" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "En la anterior serie, dejamos que pandas colocara los indices de manera automática, ahora se va a hacer de manera \"manual\"" ] }, { "cell_type": "code", "execution_count": 16, "metadata": { "collapsed": false }, "outputs": [ { "name": "stdout", "output_type": "stream", "text": [ "Serie con índices definidos\n", "1ro -1.136332\n", "2do -0.063574\n", "3ro 1.803323\n", "4to 1.257083\n", "dtype: float64\n", "\n" ] } ], "source": [ "# Note que ya no importe ni numpy ni pandas, puesto que ya estan cargadas al entorno. \n", "\n", "serie = pd.Series(np.random.randn(4),\n", " index = ['1ro','2do','3ro','4to'])\n", "print(u'Serie con índices definidos')\n", "print(format(serie))" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "En el siguiente ejemplo vamos a usar fechas para los indices" ] }, { "cell_type": "code", "execution_count": 17, "metadata": { "collapsed": false }, "outputs": [ { "name": "stdout", "output_type": "stream", "text": [ "Serie temporal con índices de fechas\n", "2016-10-15 0.252587\n", "2016-10-16 -0.220168\n", "2016-10-17 -0.262677\n", "2016-10-18 -0.267347\n", "2016-10-19 0.559145\n", "2016-10-20 0.937291\n", "2016-10-21 -0.685493\n", "2016-10-22 -0.947191\n", "2016-10-23 0.961502\n", "2016-10-24 0.178869\n", "2016-10-25 0.598649\n", "2016-10-26 -2.165758\n", "2016-10-27 -1.347345\n", "2016-10-28 -1.967871\n", "2016-10-29 0.393055\n", "2016-10-30 -0.529388\n", "2016-10-31 1.316892\n", "2016-11-01 -0.923217\n", "2016-11-02 0.478994\n", "2016-11-03 1.671387\n", "2016-11-04 -0.363334\n", "2016-11-05 -0.171606\n", "2016-11-06 -0.590254\n", "2016-11-07 -1.295299\n", "2016-11-08 -0.965223\n", "2016-11-09 0.502541\n", "2016-11-10 0.609455\n", "2016-11-11 0.211901\n", "2016-11-12 -0.553484\n", "2016-11-13 1.271707\n", "2016-11-14 -1.418531\n", "Freq: D, dtype: float64\n" ] } ], "source": [ "# serie(serie temporal) con índices que son fechas\n", "serie = pd.Series(np.random.randn(31),\n", " index = pd.date_range('2016/10/15', periods = 31))\n", "print(u'Serie temporal con índices de fechas')\n", "print(serie)" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "Como se dijo anteriormente una serie puede ser hecha de cualquier cosa" ] }, { "cell_type": "code", "execution_count": 24, "metadata": { "collapsed": false }, "outputs": [ { "name": "stdout", "output_type": "stream", "text": [ "Serie a partir de una lista\n", "0 0\n", "1 1\n", "2 4\n", "3 9\n", "4 16\n", "5 25\n", "6 36\n", "7 49\n", "8 64\n", "9 81\n", "dtype: int64\n", "__________________________________________________\n", "Serie a partir de un diccionario\n", "cuadrado de 0 0\n", "cuadrado de 1 1\n", "cuadrado de 2 4\n", "cuadrado de 3 9\n", "cuadrado de 4 16\n", "cuadrado de 5 25\n", "cuadrado de 6 36\n", "cuadrado de 7 49\n", "cuadrado de 8 64\n", "cuadrado de 9 81\n", "dtype: int64\n", "__________________________________________________\n", "Serie a partir de los valores de otra (pandas)serie \n", "0 0\n", "1 1\n", "2 4\n", "3 9\n", "4 16\n", "5 25\n", "6 36\n", "7 49\n", "8 64\n", "9 81\n", "dtype: int64\n", "__________________________________________________\n", "Serie a partir de un valor constante\n", "0 -1234\n", "1 -1234\n", "2 -1234\n", "3 -1234\n", "4 -1234\n", "5 -1234\n", "6 -1234\n", "7 -1234\n", "8 -1234\n", "9 -1234\n", "dtype: int64\n" ] } ], "source": [ "serie_lista = pd.Series([i*i for i in range(10)])\n", "print('Serie a partir de una lista')\n", "print(serie_lista)\n", "print('__________________________________________________')\n", "\n", "dicc = {'cuadrado de {}'.format(i) : i*i for i in range(10)}\n", "serie_dicc = pd.Series(dicc)\n", "print('Serie a partir de un diccionario')\n", "print(serie_dicc)\n", "print('__________________________________________________')\n", "\n", "serie_serie = pd.Series(serie_dicc.values)\n", "print('Serie a partir de los valores de otra (pandas)serie ')\n", "print(serie_serie)\n", "print('__________________________________________________')\n", "serie_cte = pd.Series(-1234, index = np.arange(10))\n", "print('Serie a partir de un valor constante')\n", "print(serie_cte)" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "Podemos llamar los elementos de las series como si fuera un diccionario o un arreglo\n" ] }, { "cell_type": "code", "execution_count": 33, "metadata": { "collapsed": false }, "outputs": [ { "name": "stdout", "output_type": "stream", "text": [ "25\n", "81\n" ] } ], "source": [ "print(serie_dicc['cuadrado de 5'])\n", "print(serie_lista[9])" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "Las operaciones están 'vectorizadas' y se hacen elemento a elemento con los elementos alineados en función del índice. Si se hace, por ejemplo, una suma de dos series, si en una de las dos series no existe un elemento, entonces el resultado para ese índice será nan." ] }, { "cell_type": "code", "execution_count": 35, "metadata": { "collapsed": false }, "outputs": [ { "name": "stdout", "output_type": "stream", "text": [ "0 0.0\n", "1 3.0\n", "2 8.0\n", "3 15.0\n", "4 24.0\n", "5 35.0\n", "6 48.0\n", "7 63.0\n", "8 NaN\n", "9 NaN\n", "dtype: float64\n" ] } ], "source": [ "X= pd.Series([i*i for i in range(10)])\n", "Y= pd.Series([j+j for j in range(8)])\n", "print(X+Y)" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "### DataFrame" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "Un DataFrame se puede ver como si fuera una tabla con índices para las filas y las columnas" ] }, { "cell_type": "code", "execution_count": 44, "metadata": { "collapsed": false }, "outputs": [ { "name": "stdout", "output_type": "stream", "text": [ "DataFrame a partir de un diccionario de listas\n", " a b\n", "0 11 21\n", "1 12 22\n", "2 13 23\n" ] } ], "source": [ "# cada columna es separada por comas\n", "df_lista = pd.DataFrame({'a': [11,12,13], 'b': [21,22,23]})\n", "print('DataFrame a partir de un diccionario de listas')\n", "print(df_lista)" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "También se pueden generar a partit de otras estructuras de python" ] }, { "cell_type": "code", "execution_count": 41, "metadata": { "collapsed": false, "scrolled": true }, "outputs": [ { "name": "stdout", "output_type": "stream", "text": [ "DataFrame a partir de un diccionario de un ndarrays\n", " a b\n", "0 0 -0.217069\n", "1 1 0.257260\n", "2 4 0.151527\n" ] } ], "source": [ "df_np = pd.DataFrame({'a': np.arange(3)**2, 'b': np.random.randn(3)})\n", "print('DataFrame a partir de un diccionario de un ndarrays')\n", "print(df_np)" ] }, { "cell_type": "code", "execution_count": 53, "metadata": { "collapsed": false }, "outputs": [ { "name": "stdout", "output_type": "stream", "text": [ "DataFrame a partir de un 2D ndarray\n", " velocidad temperatura presion\n", "primero 0.0 0.0 0.0\n", "segundo 0.0 0.0 0.0\n", "tercero 0.0 0.0 0.0\n", "cuarto 0.0 0.0 0.0\n", "quinto 0.0 0.0 0.0\n" ] } ], "source": [ "df_np2D = pd.DataFrame(np.empty((5,3)),index = ['primero','segundo','tercero','cuarto','quinto'],columns = ['velocidad', 'temperatura','presion'])\n", "print('DataFrame a partir de un 2D ndarray')\n", "print(df_np2D)" ] }, { "cell_type": "code", "execution_count": 54, "metadata": { "collapsed": false }, "outputs": [ { "name": "stdout", "output_type": "stream", "text": [ "DataFrame a partir de los valores de otro (pandas)DataFrame\n", " velocidad temperatura presion\n", "first 0.0 0.0 0.0\n", "second 0.0 0.0 0.0\n", "third 0.0 0.0 0.0\n" ] } ], "source": [ "df_df = pd.DataFrame(df_np2D, index = ['primero','segundo','tercero'])\n", "df_df.index = ['first','second','third']\n", "print('DataFrame a partir de los valores de otro (pandas)DataFrame')\n", "print(df_df)" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "## Lectura y Escritura de Archivos\n" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "Pandas es capaz de leer datos de ficheros csv, excel, HDF5, sql, json, html,..." ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "Para practicar los datos descargados de esta pagina\n", "http://www.sample-videos.com/download-sample-csv.php" ] }, { "cell_type": "code", "execution_count": 65, "metadata": { "collapsed": false }, "outputs": [ { "data": { "text/plain": [ "['.ipynb_checkpoints',\n", " 'new_func.py',\n", " 'Pandas.ipynb',\n", " 'registrations_oct-22.xlsx',\n", " 'Repaso-Algoritmos_y_Programacion.ipynb',\n", " 'resultado.txt',\n", " 'Untitled.ipynb',\n", " '__pycache__']" ] }, "execution_count": 65, "metadata": {}, "output_type": "execute_result" } ], "source": [ "os.listdir()" ] }, { "cell_type": "code", "execution_count": 104, "metadata": { "collapsed": false }, "outputs": [], "source": [ "xl = pd.ExcelFile(\"registrations_oct-22.xlsx\")" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "Aquí lo que se hizo fué cargar los datos del archivo excel en el objeto x1, lo siguiente es ver cuales hojas tiene (en este caso 1)" ] }, { "cell_type": "code", "execution_count": 105, "metadata": { "collapsed": false }, "outputs": [ { "data": { "text/plain": [ "['Sheet1']" ] }, "execution_count": 105, "metadata": {}, "output_type": "execute_result" } ], "source": [ "xl.sheet_names" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "Ahora cargo los datos de la hoja, y muestro la cabecera" ] }, { "cell_type": "code", "execution_count": 106, "metadata": { "collapsed": false }, "outputs": [ { "data": { "text/html": [ "
\n", "\n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", "
IDNameTitleFirst NameLast NameEmail AddressAffiliationPositionCountryContribution_typeContribution abstractContribution titleFunding supportAddressPhone NumberRegistration dateRegistration state
033Alexis AguirreNaNAlexisAguirrealejoda67@hotmail.comUniversidad de NariñoNaNColombiaPosterSe presenta un tratamiento canónico de la ecua...Tratamiento canónico de la ecuación de ProcaYesNaNNaN30 Aug 2016, 23:36Completed
17Aftab AhmadDr.AftabAhmadaftab.gu@gmail.comDepartment of Physics, Gomal University, Pakis...Lecturer (Gomal University, Pakistan) and rese...PakistanTalkWe evaluate the impact of an external magnetic...Inverse magnetic catalysis and confinement wit...YesNaNNaN1 Jul 2016, 18:52Completed
213Heisman Duvolfan Arcila ArboledaMr.Heisman DuvolfanArcila Arboledaheisman.arcila@udea.edu.coUniversidad de AntioquiastudentColombiaNaNNaNNaNYesNaNNaN28 Jul 2016, 22:01Completed
337Danilo Alejandro Arturo RodríguezMr.Danilo AlejandroArturo Rodríguezdanilolead1109@gmail.comUniversidad de NariñoStudentColombiaNaN.NaNYesNaNNaN9 Sep 2016, 17:03Completed
472Diego BarónMr.DiegoBaróndiegoa_baronm@hotmail.comUniversidad de AntioquiaCathedra ProffesorColombiaPosterIn our work we show how to calculate the Sagna...Sagnac Effect with path integrals.YesNaNNaN20 Oct 2016, 22:29Completed
\n", "
" ], "text/plain": [ " ID Name Title First Name \\\n", "0 33 Alexis Aguirre NaN Alexis \n", "1 7 Aftab Ahmad Dr. Aftab \n", "2 13 Heisman Duvolfan Arcila Arboleda Mr. Heisman Duvolfan \n", "3 37 Danilo Alejandro Arturo Rodríguez Mr. Danilo Alejandro \n", "4 72 Diego Barón Mr. Diego \n", "\n", " Last Name Email Address \\\n", "0 Aguirre alejoda67@hotmail.com \n", "1 Ahmad aftab.gu@gmail.com \n", "2 Arcila Arboleda heisman.arcila@udea.edu.co \n", "3 Arturo Rodríguez danilolead1109@gmail.com \n", "4 Barón diegoa_baronm@hotmail.com \n", "\n", " Affiliation \\\n", "0 Universidad de Nariño \n", "1 Department of Physics, Gomal University, Pakis... \n", "2 Universidad de Antioquia \n", "3 Universidad de Nariño \n", "4 Universidad de Antioquia \n", "\n", " Position Country \\\n", "0 NaN Colombia \n", "1 Lecturer (Gomal University, Pakistan) and rese... Pakistan \n", "2 student Colombia \n", "3 Student Colombia \n", "4 Cathedra Proffesor Colombia \n", "\n", " Contribution_type Contribution abstract \\\n", "0 Poster Se presenta un tratamiento canónico de la ecua... \n", "1 Talk We evaluate the impact of an external magnetic... \n", "2 NaN NaN \n", "3 NaN . \n", "4 Poster In our work we show how to calculate the Sagna... \n", "\n", " Contribution title Funding support \\\n", "0 Tratamiento canónico de la ecuación de Proca Yes \n", "1 Inverse magnetic catalysis and confinement wit... Yes \n", "2 NaN Yes \n", "3 NaN Yes \n", "4 Sagnac Effect with path integrals. Yes \n", "\n", " Address Phone Number Registration date Registration state \n", "0 NaN NaN 30 Aug 2016, 23:36 Completed \n", "1 NaN NaN 1 Jul 2016, 18:52 Completed \n", "2 NaN NaN 28 Jul 2016, 22:01 Completed \n", "3 NaN NaN 9 Sep 2016, 17:03 Completed \n", "4 NaN NaN 20 Oct 2016, 22:29 Completed " ] }, "execution_count": 106, "metadata": {}, "output_type": "execute_result" } ], "source": [ "df = xl.parse(\"Sheet1\")\n", "df.head()" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "De acá en adelante pueden seguir explorando, utilizando el operador \".\" sobre el objeto Hoja, o sobre el objeto excel. Por ejemplo leer la columna Contribution_type para ver cuales de ellos son presentan poster." ] }, { "cell_type": "code", "execution_count": 107, "metadata": { "collapsed": false }, "outputs": [], "source": [ "indice=[] #Para guardar los indices de los estudiantes\n", "for i in range(df.Contribution_type.first_valid_index(),df.Contribution_type.last_valid_index()):\n", " if df.Contribution_type.get_value(i) == \"Poster\":\n", " indice.append(i)\n", " " ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "### Actividad " ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "1) De los asistentes que presentan poster, cuales son los nombres de los que vivien en Colombia?\n", "\n", "2) y cuantos asistentes Colombianos con poster piden ayuda." ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "Podemos guardar en otro archivo lo que queramos usando el operador \".to_excel\", hay muchas otras posibilidades." ] }, { "cell_type": "code", "execution_count": 109, "metadata": { "collapsed": true }, "outputs": [], "source": [ "df.to_excel('Titulo.xlsx')" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "## Utilidades Generales " ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "Generemos un set de datos aleatorio de prueba" ] }, { "cell_type": "code", "execution_count": 114, "metadata": { "collapsed": false }, "outputs": [ { "name": "stdout", "output_type": "stream", "text": [ " velocidad temperatura presion\n", "primero -0.767273 1.061980 -0.657995\n", "segundo 1.093620 0.500940 -2.163514\n", "tercero -0.705949 0.063516 -0.316575\n", "cuarto -0.033902 0.588237 0.292650\n", "quinto 0.406155 -1.425972 -1.358436\n" ] } ], "source": [ "df = pd.DataFrame(np.random.randn(5,3),\n", " index = ['primero','segundo','tercero','cuarto','quinto'],\n", " columns = ['velocidad', 'temperatura','presion'])\n", "print(df)\n" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "A estos datos podemos adicionar, una columna." ] }, { "cell_type": "code", "execution_count": 115, "metadata": { "collapsed": false }, "outputs": [ { "name": "stdout", "output_type": "stream", "text": [ " velocidad temperatura presion velocidad_maxima\n", "primero -0.767273 1.061980 -0.657995 -1.535681\n", "segundo 1.093620 0.500940 -2.163514 -0.609931\n", "tercero -0.705949 0.063516 -0.316575 -0.355511\n", "cuarto -0.033902 0.588237 0.292650 0.614520\n", "quinto 0.406155 -1.425972 -1.358436 -0.076098\n" ] } ], "source": [ "df['velocidad_maxima'] = np.random.randn(df.shape[0])\n", "print(df)" ] }, { "cell_type": "raw", "metadata": {}, "source": [ "Pero qué pasa si quiero añadir la columna en un lugar específico. Para ello podemos usar el método insert (también sirve para borrar una columna):" ] }, { "cell_type": "code", "execution_count": 116, "metadata": { "collapsed": true }, "outputs": [], "source": [ "#Guardamos la columna en una variable, y luego la borramos del arreglo\n", "columna = df['velocidad_maxima']\n", "del df['velocidad_maxima']" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "Ahora la incertamos en la posición que quiero" ] }, { "cell_type": "code", "execution_count": 117, "metadata": { "collapsed": false }, "outputs": [ { "name": "stdout", "output_type": "stream", "text": [ " velocidad velocidad_maxima temperatura presion\n", "primero -0.767273 -1.535681 1.061980 -0.657995\n", "segundo 1.093620 -0.609931 0.500940 -2.163514\n", "tercero -0.705949 -0.355511 0.063516 -0.316575\n", "cuarto -0.033902 0.614520 0.588237 0.292650\n", "quinto 0.406155 -0.076098 -1.425972 -1.358436\n" ] } ], "source": [ "df.insert(1, 'velocidad_maxima', columna)\n", "print(df)" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "Para que sirven estos comandos:\n", "\n", "1) df.ix['tercero':'quinto']\n", "\n", "2) df.xs('tercero')\n", "\n", "3) df[df['temperatura' > 0]['velocidad']\n", "\n", "4) Investiguen los métodos .loc(), .iloc(), .select()" ] }, { "cell_type": "code", "execution_count": null, "metadata": { "collapsed": true }, "outputs": [], "source": [] } ], "metadata": { "anaconda-cloud": {}, "kernelspec": { "display_name": "Python [default]", "language": "python", "name": "python3" }, "language_info": { "codemirror_mode": { "name": "ipython", "version": 3 }, "file_extension": ".py", "mimetype": "text/x-python", "name": "python", "nbconvert_exporter": "python", "pygments_lexer": "ipython3", "version": "3.5.2" } }, "nbformat": 4, "nbformat_minor": 1 }