Image by Author | Canva When it comes to error handling, the first thing we usually learn is how to use try-except blocks. But is that really enough as our codebase grows more complex? I believe not. Relying solely on try-except can lead to repetitive, cluttered, and hard-to-maintain code.
In this article, I’ll walk you through 5 advanced yet practical error handling patterns that can make your code cleaner, more reliable, and easier to debug. Each pattern comes with a real-world example so you can clearly see where and why it makes sense. So, let’s get started.
1. Error Aggregation for Batch Processing
When processing multiple items (e.g., in a loop), you might want to continue processing even if some items fail, then report all errors at the end. This pattern, called error aggregation, avoids stopping on the first failure. This pattern is excellent for form validation, data import scenarios, or any situation where you want to provide comprehensive feedback about all issues rather than stopping at the first error.
Example: Processing a list of user records. Continue even if some fail.
def process_user_record(record, record_number):
if not record.get("email"):
raise ValueError(f"Record #{record_number} failed: Missing email in record {record}")
# Simulate processing
print(f"Processed user #{record_number}: {record['email']}")
def process_users(records):
errors = []
for index, record in enumerate(records, start=1):
try:
process_user_record(record, index)
except ValueError as e:
errors.append(str(e))
return errors
users = [
{"email": "qasim@example.com"},
{"email": ""},
{"email": "zeenat@example.com"},
{"email": ""}
]
errors = process_users(users)
if errors:
print("\nProcessing completed with errors:")
for error in errors:
print(f"- {error}")
else:
print("All records processed successfully")
This code loops through user records and processes each one individually. If a record is missing an email, it raises a ValueError, which is caught and stored in the errors list. The process continues for all records, and any failures are reported at the end without stopping the entire batch like this:
Output:
Processed user #1: qasim@example.com
Processed user #3: zeenat@example.com
Processing completed with errors:
- Record #2 failed: Missing email in record {'email': ''}
- Record #4 failed: Missing email in record {'email': ''}
2. Context Manager Pattern for Resource Management
When working with resources like files, database connections, or network sockets, you need to ensure they’re properly opened and closed, even if an error occurs. Context managers, using the with statement, handle this automatically, reducing the chance of resource leaks compared to manual try-finally blocks. This pattern is especially helpful for I/O operations or when dealing with external systems.
Example: Let’s say you’re reading a CSV file and want to ensure it’s closed properly, even if processing the file fails.
import csv
def read_csv_data(file_path):
try:
with open(file_path, 'r') as file:
print(f"Inside 'with': file.closed = {file.closed}") # Should be False
reader = csv.reader(file)
for row in reader:
if len(row) < 2:
raise ValueError("Invalid row format")
print(row)
print(f"After 'with': file.closed = {file.closed}") # Should be True
except FileNotFoundError:
print(f"Error: File {file_path} not found")
print(f"In except block: file is closed? {file.closed}")
except ValueError as e:
print(f"Error: {e}")
print(f"In except block: file is closed? {file.closed}")
# Create test file
with open("data.csv", "w", newline="") as f:
writer = csv.writer(f)
writer.writerows([["Name", "Age"], ["Sarwar", "30"], ["Babar"], ["Jamil", "25"]])
# Run
read_csv_data("data.csv")
This code uses a with statement (context manager) to safely open and read the file. If any row has fewer than 2 values, it raises a ValueError, but the file still gets closed automatically. The file.closed checks confirm the file’s state both inside and after the with block—even in case of an error. Let’s run the above code to observe this behavior:
Output:
Inside 'with': file.closed = False
['Name', 'Age']
['Sarwar', '30']
Error: Invalid row format
In except block: file is closed? True
3. Exception Wrapping for Contextual Errors
Sometimes, an exception in a lower-level function doesn’t provide enough context about what went wrong in the broader application. Exception wrapping (or chaining) lets you catch an exception, add context, and re-raise a new exception that includes the original one. It’s especially useful in layered applications (e.g., APIs or services).
Example: Suppose you’re fetching user data from a database and want to provide context when a database error occurs.
class DatabaseAccessError(Exception):
"""Raised when database operations fail."""
pass
def fetch_user(user_id):
try:
# Simulate database query
raise ConnectionError("Failed to connect to database")
except ConnectionError as e:
raise DatabaseAccessError(f"Failed to fetch user {user_id}") from e
try:
fetch_user(123)
except DatabaseAccessError as e:
print(f"Error: {e}")
print(f"Caused by: {e.__cause__}")
The ConnectionError is caught and wrapped in a DatabaseAccessError with additional context about the user ID. The from e syntax links the original exception, so the full error chain is available for debugging. The output might look like this:
Output:
Error: Failed to fetch user 123
Caused by: Failed to connect to database
4. Retry Logic for Transient Failures
Some errors, like network timeouts or temporary service unavailability, are transient and may resolve on retry. Using a retry pattern can handle these gracefully without cluttering your code with manual loops. It automates recovery from temporary failures.
Example: Let’s retry a flaky API call that occasionally fails due to simulated network errors. The code below attempts the API call multiple times with a fixed delay between retries. If the call succeeds, it returns the result immediately. If all retries fail, it raises an exception to be handled by the caller.
import random
import time
def flaky_api_call():
# Simulate 50% chance of failure (like timeout or server error)
if random.random() < 0.5:
raise ConnectionError("Simulated network failure")
return {"status": "success", "data": [1, 2, 3]}
def fetch_data_with_retry(retries=4, delay=2):
attempt = 0
while attempt < retries:
try:
result = flaky_api_call()
print("API call succeeded:", result)
return result
except ConnectionError as e:
attempt += 1
print(f"Attempt {attempt} failed: {e}. Retrying in {delay} seconds...")
time.sleep(delay)
raise ConnectionError(f"All {retries} attempts failed.")
try:
fetch_data_with_retry()
except ConnectionError as e:
print("Final failure:", e)
Output:
Attempt 1 failed: Simulated network failure. Retrying in 2 seconds...
API call succeeded: {'status': 'success', 'data': [1, 2, 3]}
As you can see, the first attempt failed due to the simulated network error (which happens randomly 50% of the time). The retry logic waited for 2 seconds and then successfully completed the API call on the next attempt.
5. Custom Exception Classes for Domain-Specific Errors
Instead of relying on generic exceptions like ValueError or RuntimeError, you can create custom exception classes to represent specific errors in your application’s domain. This makes error handling more semantic and easier to maintain.
Example: Suppose a payment processing system where different types of payment failures need specific handling.
class PaymentError(Exception):
"""Base class for payment-related exceptions."""
pass
class InsufficientFundsError(PaymentError):
"""Raised when the account has insufficient funds."""
pass
class InvalidCardError(PaymentError):
"""Raised when the card details are invalid."""
pass
def process_payment(amount, card_details):
try:
if amount > 1000:
raise InsufficientFundsError("Not enough funds for this transaction")
if not card_details.get("valid"):
raise InvalidCardError("Invalid card details provided")
print("Payment processed successfully")
except InsufficientFundsError as e:
print(f"Payment failed: {e}")
# Notify user to top up account
except InvalidCardError as e:
print(f"Payment failed: {e}")
# Prompt user to re-enter card details
except Exception as e:
print(f"Unexpected error: {e}")
# Log for debugging
process_payment(1500, {"valid": False})
Custom exceptions (InsufficientFundsError, InvalidCardError) inherit from a base PaymentError class, allowing you to handle specific payment issues differently while catching unexpected errors with a generic Exception block. For example, In the call process_payment(1500, {“valid”: False}), the first check triggers because the amount (1500) exceeds 1000, so it raises InsufficientFundsError. This exception is caught in the corresponding except block, printing:
Output:
Payment failed: Not enough funds for this transaction
Conclusion
That’s it. In this article, we explored 5 practical error handling patterns:
- Error Aggregation: Process all items, collect errors, and report them together
- Context Manager: Safely manage resources like files with with blocks
- Exception Wrapping: Add context by catching and re-raising exceptions
- Retry Logic: Automatically retry transient errors like network failures
- Custom Exceptions: Create specific error classes for clearer handling
Give these patterns a try in your next project. With a bit of practice, you’ll find your code easier to maintain and your error handling much more effective.
Kanwal Mehreen Kanwal is a machine learning engineer and a technical writer with a profound passion for data science and the intersection of AI with medicine. She co-authored the ebook “Maximizing Productivity with ChatGPT”. As a Google Generation Scholar 2022 for APAC, she champions diversity and academic excellence. She’s also recognized as a Teradata Diversity in Tech Scholar, Mitacs Globalink Research Scholar, and Harvard WeCode Scholar. Kanwal is an ardent advocate for change, having founded FEMCodes to empower women in STEM fields.
