Big Data

Exception handling in Python | Test and Except in Python

Introduction

Errors are the nightmare of a programmer's existence. You write an amazing piece of code, you're ready to run it and build a powerful machine learning model, and then poof. Python throws an unexpected error, ending your hope of quick code execution.

Each and every one of us has faced this problem and came out of it a better programmer.. Facing mistakes and mistakes is what builds our long-term confidence and teaches us valuable lessons along the way..

We have some rules when writing programs in any programming language, how not to use a space when defining a variable name, add colon (:) after the if statement, and so on. If we don't follow these rules, we run into syntax errors and our program refuses to run until we remove those errors.

But there are times when the program is syntactically correct and still throws an error when we try to run the program. What's going on here? Good, these errors caught during execution are called exceptions. And dealing with these errors is called exception handling.

We will talk about exception handling in Piton here!

Advantages of exception handling
Common exceptions
Statement of evidence and exception
Other statement
Finally statement
Real life use case of exception handling

Advantages of exception handling

Why should you learn to handle exceptions? Let me answer that using a two-pronged argument:

Suppose you have written a script to read thousands of files present in various directories. Now, there may be some kind of error there, like a file type is missing or is wrong format or is available in different extensions. Then, not possible to open all files and write script accordingly. Exception handling allows us to define multiple conditions. For instance, if the format is wrong, fix it first then try to read the file. On the contrary, skip reading that file and create a log file so we can deal with it later.
Secondly, let's say we are pulling restaurant data from a website and our script looks for the name, the restaurant's reviews and address. For some reason, the restaurant's address is missing on the website. In this case, if we are not handling exceptions, our script can stop in the middle, so while collecting data in a large quantity, it is essential that we handle exceptions.

Common exceptions

Here is a list of the common exceptions that you will find in Python:

ZeroDivisionError: It rises when you try to divide a number by zero
ImportError: Raised when you try to import the library which is not installed or you have supplied the wrong name
IndexError: Raised when no index is found in a sequence. For instance, if the length of the list is 10 and you are trying to access the eleventh index of that list, you will get this error.
IndentationError: Raised when indentation is not specified correctly
ValueError: Raised when the built-in function for a data type has the valid type of arguments, but the arguments have invalid values specified
Exception: Base class for all exceptions. If you are not sure which exception may occur, you can use base class. Will take care of all of them

You can read about more common exceptions here.

Exception handling in Python: test and exception statement

Let's define a function to divide two numbers a Y B. It will work fine if the value of b is nonzero, but it will generate an error if the value of b is zero:

We can handle this using the treat Y except statement. First, try clause will be executed, which are the declarations between the try and except keywords.

If no exception occurs, the except clause will be omitted. Secondly, if an exception occurs during the execution of the try clause, the rest of the try statements will be ignored:

Exception handling in Python: Else statement

In Python, We can also instruct a program to execute certain lines of code if no exception occurs using the the rest clause. Now, if no exception occurs in the above code, we want to print “No error occurred !!”.

Let's see how to do this:

Exception handling in Python: declaration finally

Now, What if we need some kind of action to be executed whether the error occurred or not (how to keep records)? For this, we have the Finally clause in Python. It will always run whether the program gets any of the exceptions or not.

We will see how we can use the Finally clause to write the records later in this article.

Now, in the example above, I want to print the value of a Y B after each run, regardless of whether the error occurred or not. Let's see how to do that:

Real life use case of exception handling in Python

Up to now, we have seen exception handling on some random data. How about raising the lever a bit and understanding this using a real life example?

We have data containing the details of the employees, like your education, age, number of trainings carried out, etc. The data is divided into several directories per region. Details of employees belonging to the same region are stored in the same file.

Now, our task is to read all the files and concatenate them to form a single file. Let's start by importing some of the necessary libraries.

Get the directory structure

To view the directory structure, we will use the globe library and to read CSV files, we will use the Pandas Library:

View the directory structure using the glob.glob function and destination directory path. You can download the directory structure here.

We can see that the names of the folders are represented as some numbers and in the next step we will go through each of the directories and we will see the files present:

In each of the folders, there is a CSV file present that contains the details of the employees of that particular region. You can open and view any CSV file. Below is the picture of how the data looks in the region_1.csv proceedings. Contains details of employees belonging to the region 1:

Read the files

Now, we know there is a pattern in the directory structure and filename. In the address book North, there is a CSV file name region_n present. So now we will try to read all these files using a loop.

We know that the maximum number is 34, so we will use a in loop to iterate and read the files in order:

You can see that the file region_7 is not present. Then, one of the easiest ways to deal with this is to put a and condition in the program – if the directory name is 7 then skip reading that file.

But, What if we have to read thousands of files together? It would be a tedious task to update the if condition every time we get an error.

Here, we will use the try and except statement to handle errors. If there are any exceptions during runtime while reading any file, we will just skip that step and continue reading the next folder. We will print the file name with “File not found!” if the error is FileNotFoundError and print the file name with “Other error!” if any other error occurs.

We can see that we got another error in file number 32. Let's try to read this file separately:

There are some problems with the File format 32. If you open the region_32.csv file, you will see that there are some comments added on top of the file and we can read that file using the jumps parameter:

Let's see how to handle this.

Try statements, Except y Else

We will create two Boolean variables: parse error, file not found – and initialize both to False at the beginning of each iteration. Then, if we get FileNotFoundError, then we will establish file not found as True.

Later, we will print that particular file as missing and skip that iteration. If we succeed ParserError, then we will establish parse error as True and print that the particular file is in the wrong format and re-read the file using the jumps parameter.

Now, if no exception occurs, the code of the else statement will be executed. In the else statement, we will add the data frame to the list of data frames:

Test statement, exception, another and finally

Suppose you want to create a log file to keep track of which files are correct and which have errors. This is a use case of the finally statement. Whether I get the error or not, the sentence will finally be executed.

Then in the Finally clause, we will write to the file regarding the state of the file at each iteration:

Final notes

I use exception handling when extracting data from multiple web pages. I would love to know how you use exception handling, so do comment below with your thoughts and share them with the community.

If you found this article informative, Share it with your friends and comment below on your questions and comments. I have listed some amazing Python and data science related articles below for your reference: