This post was made public as part of the Data Science Blogathon.
Introduction
Python is easy to learn, has a large online community of students and instructors, and has some truly powerful data-centric libraries. Pandas is one of the most important Python libraries for data analysis and data science.
In this post, we will see the 13 Most important Pandas functions and methods that are essential for all analysts and data scientists to know.
1. read_csv ()
The read_csv function () helps to read a comma separated values file (csv) in a Pandas DataFrame. All you need to do is mention the path of the file you want it to read. It can also read files separated by delimiters other than commas, What | or tab. More details here.
data_1 = pd.read_csv(r'C:UsersABCDesktopblog_dataset.csv')
Los datos se han leído desde la Data SourceA "Data Source" refers to any place or medium where information can be obtained. These sources can be both primary and, such as surveys and experiments, as secondary, as databases, academic articles or statistical reports. The right choice of a data source is crucial to ensure the validity and reliability of information in research and analysis.... en Pandas DataFrame. You will need to change the path of the file you want to read. You can download the dataset used in the blog.
The to_csv function () works exactly opposite to read_csv (). Helps to write data contained in Pandas DataFrame or Series to csv file. You can read more about to_csv () here. read_csv () y to_csv () they are one of the most used functions in Pandas because they are used when reading data from a data source, and it is very important to know them.
2. head ()
head (n) is used to return the first n rows of a data set. By default, df.head () will return the first 5 rows of the DataFrame. If you want more / less number of rows, you can specify n as an integer.
data_1.head(6)
Production:
| Name | Age | Town | Condition | DOB | Gender | City temperature | Salary | |
|---|---|---|---|---|---|---|---|---|
| 0 | Nature | 29 | Indore | Madhya Pradesh | 20-11-1991 | Masculine | 35,5 | 50000 |
| 1 | Rohit | 23 | New Delhi | Delhi | 19-09-1997 | Masculine | 39,0 | 85000 |
| 2 | Bimla | 35 | Rohtak | Haryana | 09-01-1985 | Woman | 39,7 | 20000 |
| 3 | Rahul | 25 | Calcutta | west bengal | 19-09-1995 | Masculine | 36,5 | 40000 |
| 4 | Chaman | 32 | Chennai | Tamil Nadu | 12-03-1988 | Masculine | 41,1 | 65000 |
| 5 | Vivek | 38 | Gurugram | Haryana | 22-06-1982 | Masculine | 38,9 | 35000 |
The first 6 rows (indexed from 0 a 5) are returned as output as per expectations.
tail () is equivalent to head () and returns the bottom n rows of a data set. head () y tail () help you to take a quick look at your dataset and check if the data has been read correctly from the DataFrame.
3. describe ()
describe () is used to generate descriptive statistics of the data in a DataFrame or Pandas Series. Summarizes the central tendency and dispersion of the data set. describe () helps to get a quick overview of the dataset. More details can be found on describe () here.
data_1.describe()
Production:
| Age | City temperature | Salary | |
|---|---|---|---|
| tell | 9.000000 | 8.000000 | 9.000000 |
| to mean | 32.000000 | 38.575000 | 44444.444444 |
| std | 5.894913 | 1.771803 | 21360.659582 |
| min | 23.000000 | 35.500000 | 18000.000000 |
| 25% | 29.000000 | 38.300000 | 35000.000000 |
| 50% | 32.000000 | 38.950000 | 40000.000000 |
| 75% | 38.000000 | 39.175000 | 52000.000000 |
| max | 39.000000 | 41.100000 | 85000.000000 |
describe () lists different descriptive statistical measures for all numeric columns in our dataset. By assigning the inclusion attribute the value 'all', we can get the description to include all columns, including those containing categorical information.
4. memory_use ()
memory_usage () returns a Pandas string that has the memory usage of each column (and bytes) in a Pandas DataFrame. By specifying the deep attribute as True, we can know the real space that each column occupies. More details can be found on memory_usage () here.
data_1.memory_usage(deep=True)
Production:
Index 80 Name 559 Age 72 City 578 State 584 DOB 603 Gender 553 City temp 72 Salary 72 dtype: int64
The memory usage of each column has been output in a Pandas series. It is essential to know the memory usage of a DataFrame, so that it can address errors like MemoryError in Python.
5. astype ()
astype () is used to convert a Python object to a particular data type. It can be a very useful feature in case your data is not stored in the correct format (type of data). As an example, if python has somehow misinterpreted floating point numbers as strings, you can convert them back to floating point numbers with astype (). Or if you want to convert an object data type to a category, you can use astype ().
data_1['Gender'] = data_1.Gender.astype('category')
You can check the data type change by looking at the data types of all the columns in the dataset using the dtypes attribute. To view the astype documentation (), click on here.
6. place[:]
place[:] helps to enter a group of rows and columns in a data set, a portion of the data set, according to our requirement. As an example, if we only want the last 2 rows and the first 3 columns of a dataset, we can enter them with the help of loc[:]. We can also enter rows and columns supported by labels instead of rows and column numbers.
data_1.loc[0:4, ['Name', 'Age', 'State']]
Production:
| Name | Age | Condition | |
|---|---|---|---|
| 0 | Nature | 29 | Madhya Pradesh |
| 1 | Rohit | 23 | Delhi |
| 2 | Bimla | 35 | Haryana |
| 3 | Rahul | 25 | west bengal |
| 4 | Chaman | 32 | Tamil Nadu |
The above code will return the columns “Name”, “Age” Y “Condition” for the first 5 customer records. Keep in mind that the indexThe "Index" It is a fundamental tool in books and documents, which allows you to quickly locate the desired information. Generally, it is presented at the beginning of a work and organizes the contents in a hierarchical manner, including chapters and sections. Its correct preparation facilitates navigation and improves the understanding of the material, making it an essential resource for both students and professionals in various areas.... comienza desde 0 and Python, so what[:] it is inclusive in both mentioned values. Then 0: 4 will mean indices of 0 a 4, both included.
place[:] it is one of the most powerful functions of Pandas, and is a must-have for all data analysts and data scientists. You can find the documentation for loc[:] here.
iloc[:] works equivalently, just that iloc[:] it is not inclusive in both values. Tan iloc[0:4] would return rows with index 0, 1, 2 Y 3, while loc[0:4] would return rows with index 0, 1, 2, 3 Y 4. Documentation for iloc[:] It can be found here.
7. to_datetime ()
to_datetime () convert a python object to datetime format. Can take an integer, a floating point number, a list, Pandas Series o Pandas DataFrame as an argument. to_datetime () it is very powerful when the dataset has time series or date values.
data_1['DOB'] = pd.to_datetime(data_1['DOB'])
DOB column has now been changed to Pandas data time format. Todas las funciones de fechaDate functions are essential tools in programming and data analysis. Allows manipulation, Calculate and format dates efficiently. Among its most common applications are the calculation of the difference between dates, the extraction of components such as the year or month, and conversion between different formats. These functions are critical in areas such as project management, financial reporting and statistical analysis.... y hora ahora se pueden aplicar en esta columna. You can read more about to_datetime () here.
8. value_counts ()
value_counts () returns a Pandas string containing the counts of unique values. Consider a data set that contains customer information about 5,000 clients of a company. value_counts () will help us identify the number of occurrences of each unique value in a Series. Can be applied to columns containing data like Status, Industry of employment or age of clients.
data_1['State'].value_counts()
Production:
Haryana 3 Delhi 2 West Bengal 1 Tamil Nadu 1 Bihar 1 Madhya Pradesh 1 Name: State, dtype: int64
The number of occurrences of each state in our dataset has been returned in the output, as expected. value_counts () it can also be used to plot bar graphs of categorical and ordinal data.
data_1['State'].value_counts(normalize=True).plot(kind='bar', title="State")
The documentation for value_counts () can be found here.
9. drop_duplicates ()
drop_duplicates () returns a Pandas DataFrame with duplicate rows removed. Even among duplicates, there is the option to keep the first occurrence (Registration) of the duplicate or the last. You can also specify the inplace and ignore_index attribute.
data_1.drop_duplicates(inplace=True)
inplace = True ensures that changes are applied to the original dataset. You can verify the changes by looking at the shape of the original dataset and the modified dataset (after deleting duplicates). You will notice that the number of rows has been reduced from 9 a 8 (because it was removed 1 duplicate).
10. groupby ()
groupby () is used to group a Pandas DataFrame by 1 or more columns and perform some mathematical operation on it. groupby () can be used to summarize data in a simple way.
data_1.groupby(by='State').Salary.mean()
Production:
State Bihar 18000 Delhi 68500 Haryana 27500 Madhya Pradesh 50000 Tamil Nadu 65000 West Bengal 40000 Name: Salary, dtype: int64
The above code will group the dataset by column “Condition” and will return the average age in all states. Can click here to know more about groupby ().
11. fuse ()
merge () is used to merge 2 Pandas DataFrame objects or a DataFrame and a Series object in a column (field) common. Si está familiarizado con el concepto de JOIN"JOIN" is a fundamental operation in databases that allows you to combine records from two or more tables based on a logical relationship between them. There are different types of JOIN, as INNER JOIN, LEFT JOIN and RIGHT JOIN, each with its own characteristics and uses. This technique is essential for complex queries and more relevant and detailed information from multiple data sources.... and SQL, combine a function equivalent to that. Returns the combined DataFrame.
data_1.merge(data_2, on='Name', how='left')
For more information on attributes like on (incluidos left_on y right_on), how and suffixes, see the documentation.
12. sort_values ()
sort_values () is used to sort the column in a Pandas data frame (or a series Pandas) by values in ascending or descending order. By specifying the inplace attribute as True, you can make a change directly to the original DataFrame.
data_1.sort_values(by='Name', inplace=True)
Production:
| Name | Age | Town | Condition | DOB | Gender | City temperature | Salary | |
|---|---|---|---|---|---|---|---|---|
| 0 | Nature | 29 | Indore | Madhya Pradesh | 1991-11-20 | Masculine | 35,5 | 50000 |
| 2 | Bimla | 35 | Rohtak | Haryana | 1985-09-01 | Woman | 39,7 | 20000 |
| 4 | Chaman | 32 | Chennai | Tamil Nadu | 1988-12-03 | Masculine | 41,1 | 65000 |
| 6 | Charu | 29 | New Delhi | Delhi | 1992-03-18 | Woman | 39,0 | 52000 |
| 7 | Ganesh | 39 | Patna | Bihar | 1981-07-12 | Masculine | Yaya | 18000 |
| 3 | Rahul | 25 | Calcutta | west bengal | 1995-09-19 | Masculine | 36,5 | 40000 |
| 1 | Rohit | 23 | New Delhi | Delhi | 1997-09-19 | Masculine | 39,0 | 85000 |
| 5 | Vivek | 38 | Gurugram | Haryana | 1982-06-22 | Masculine | 38,9 | 35000 |
You can see that the order of the records has changed now. Records are now listed in alphabetical order of names. sort_values () has many other attributes that can be specified. You can read about this here.
Semejante a sort_values () es sort_index (). Used to sort the DataFrame by index instead of a column value.
13. fillna ()
As usual, in a large data set, you will find multiple entries labeled NaN by Python. NaN means “it is not a number” y represents entries that were not completed in the original data source. When filling in the values in the DataFrame, Pandas makes sure that the user can identify these inputs separately.
fillna () helps to replace all NaN values in a DataFrame or Series by imputing these missing values with more appropriate values.
data_1['City temp'].fillna(38.5, inplace=True)
The above code will replace all blank entries in “city temperature” with 38.5. Missing values can be imputed with the mean, the medianThe median is a statistical measure that represents the central value of a set of ordered data. To calculate it, the data is organized from lowest to highest and the number in the middle is identified. If there are an even number of observations, the two core values are averaged. This indicator is especially useful in asymmetric distributions, since it is not affected by extreme values...., fashion or some other value. We have chosen the medium for our case.
EndNotes
In this post, we take a look at the 13 Pandas most important functions and methods that are important for data analysis and data science. This post was written by Vishesh Arora (LinkedIn).
The media shown in this post is not the property of DataPeaker and is used at the author's discretion.
