Post

Python Notes

Posted Updated
By ainotes 33 min read

Q&A

write a logging decorator can log the entry, exit, and any exceptions of a function.

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20

a,b = 5,0

def log_decorator(func):
  def wrapper(*arg,**kwargs):
    print(f"Calling {func.__name__} with arg: {arg} and kwargs: {kwargs}")
    try:
      result = func(*arg,**kwargs)
      print(f"Result of {func.__name__}: {result}")
    except Exception as e:
      print(f"Exception in {func.__name__}: {e}")
    finally:
      print(f"Exiting {func.__name__}")
  return wrapper

@log_decorator
def divide(a,b):
  return a//b

divide(a,b)

Write an authorization decorator can check if a user has the required permissions to execute a function.

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23

import functools
def authorize(roles):
    def decorator(func):
        @functools.wraps(func)
        def wrapper(user_role, *args, **kwargs):
            if user_role not in roles:
                print(f"Access denied for role: {user_role}")
                return None
            return func(user_role, *args, **kwargs)
        return wrapper
    return decorator

@authorize(roles=['admin', 'user'])
def access_resource(user_role, resource):
    print(f"{user_role} accessing {resource}")

access_resource('admin', 'Settings')
# Output:
# admin accessing Settings

access_resource('guest', 'Settings')
# Output:
# Access denied for role: guest

Write a timing decorator can measure the execution time of a function.

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22

import time
import functools
def timing_decorator(func):
    @functools.wraps(func)
    def wrapper(*args, **kwargs):
        start_time = time.time()
        result = func(*args, **kwargs)
        end_time = time.time()
        print(f"{func.__name__} executed in {end_time - start_time:.4f} seconds")
        return result
    return wrapper

@timing_decorator
def slow_function(seconds):
    time.sleep(seconds)
    return f"Finished sleeping for {seconds} seconds"

print(slow_function(2))
# Output:
# slow_function executed in 2.000x seconds
# Finished sleeping for 2 seconds

Write a caching decorator can store the results of expensive function calls and return the cached result when the same inputs occur again.

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21

def cache_decorator(func):
    cache = {}
    @functools.wraps(func)
    def wrapper(*args):
        if args in cache:
            print(f"Returning cached result for {args}")
            return cache[args]
        result = func(*args)
        cache[args] = result
        return result
    return wrapper

@cache_decorator
def fibonacci(n):
    if n in (0, 1):
        return n
    return fibonacci(n - 1) + fibonacci(n - 2)

print(fibonacci(10))
# Output will include multiple "Returning cached result for" lines, showing the caching in action.

A retry decorator can retry a function if it raises a certain exception.

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27

def retry_decorator(retries=3, exceptions=(Exception,)):
    def decorator(func):
        @functools.wraps(func)
        def wrapper(*args, **kwargs):
            for attempt in range(retries):
                try:
                    return func(*args, **kwargs)
                except exceptions as e:
                    print(f"Attempt {attempt + 1} failed: {e}")
                    if attempt == retries - 1:
                        raise
        return wrapper
    return decorator

@retry_decorator(retries=3, exceptions=(ZeroDivisionError,))
def divide(a, b):
    return a / b

try:
    print(divide(10, 0))
except ZeroDivisionError:
    print("All attempts failed.")
# Output:
# Attempt 1 failed: division by zero
# Attempt 2 failed: division by zero
# Attempt 3 failed: division by zero
# All attempts failed.

Generators in Python are useful for efficiently managing memory and handling large or infinite sequences. Here are some advanced use cases:

1. Reading Large Files

Generators can be used to read large files line by line without loading the entire file into memory.

1
2
3
4
5
6
7

def read_large_file(file_path):
    with open(file_path, 'r') as file:
        for line in file:
            yield line

for line in read_large_file('large_file.txt'):
    process(line)  # Replace with actual processing logic

2. Generating Infinite Sequences

Generators can produce infinite sequences, useful for simulations, testing, or generating data on-the-fly.

1
2
3
4
5
6
7
8
9

def infinite_sequence(start=0):
    num = start
    while True:
        yield num
        num += 1

gen = infinite_sequence()
for _ in range(10):
    print(next(gen))  # Prints numbers from 0 to 9

3. Pipeline Processing

Generators can be used to create processing pipelines, where data flows through a series of generator functions.

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21

def generate_numbers(limit):
    for num in range(1, limit + 1):
        yield num

def square_numbers(numbers):
    for num in numbers:
        yield num * num

def sum_numbers(numbers):
    total = 0
    for num in numbers:
        total += num
        yield total

limit = 10
numbers = generate_numbers(limit)
squared_numbers = square_numbers(numbers)
summed_numbers = sum_numbers(squared_numbers)

for total in summed_numbers:
    print(total)  # Prints the running total of squares of numbers from 1 to 10

4. Producer-Consumer Model

Generators can implement the producer-consumer model, where one generator produces data and another consumes it.

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26

import time

def producer():
    for i in range(1, 6):
        print(f"Produced {i}")
        yield i
        time.sleep(1)  # Simulate time-consuming production

def consumer(generator):
    for item in generator:
        print(f"Consumed {item}")
        time.sleep(2)  # Simulate time-consuming consumption

gen = producer()
consumer(gen)
# Output:
# Produced 1
# Consumed 1
# Produced 2
# Consumed 2
# Produced 3
# Consumed 3
# Produced 4
# Consumed 4
# Produced 5
# Consumed 5

5. Generating Fibonacci Sequence

A generator can be used to generate the Fibonacci sequence efficiently.

1
2
3
4
5
6
7
8
9

def fibonacci():
    a, b = 0, 1
    while True:
        yield a
        a, b = b, a + b

fib = fibonacci()
for _ in range(10):
    print(next(fib))  # Prints the first 10 Fibonacci numbers

6. Combinatorial Generators

Generators can be used to yield combinations, permutations, or Cartesian products.

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29

import itertools

def combinations(iterable, r):
    pool = tuple(iterable)
    n = len(pool)
    if r > n:
        return
    indices = list(range(r))
    yield tuple(pool[i] for i in indices)
    while True:
        for i in reversed(range(r)):
            if indices[i] != i + n - r:
                break
        else:
            return
        indices[i] += 1
        for j in range(i + 1, r):
            indices[j] = indices[j - 1] + 1
        yield tuple(pool[i] for i in indices)

for combo in combinations('ABCD', 2):
    print(combo)
# Output:
# ('A', 'B')
# ('A', 'C')
# ('A', 'D')
# ('B', 'C')
# ('B', 'D')
# ('C', 'D')

7. Simulating Data Streams

Generators can simulate data streams for real-time data processing applications.

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15

import random
import time

def data_stream():
    while True:
        data = random.randint(0, 100)
        yield data
        time.sleep(1)  # Simulate data arrival every second

for data in data_stream():
    print(f"Received data: {data}")
    if data > 90:
        print("Data is above threshold, stopping...")
        break
# Output will vary but will simulate data arriving and being processed.

These advanced examples illustrate how generators can be used for efficient memory management, creating infinite or large sequences, building processing pipelines, and more. They demonstrate the power and versatility of generators in Python for handling various real-world scenarios.

Python Notes

Data Structure

NameTypeDescription
IntegersintWhole numbers, such as: 3 300 200
Floating pointfloatNumbers with a decimal point: 2.3 4.6 100.0
StringsstrOrdered sequence of characters: “hello” ‘Sammy’ “2000” “???”
ListslistOrdered sequence of objects: [10,”hello”,200.3]
DictionariesdictUnordered Key:Value pairs: {“mykey” : “value” , “name” : “Frankie”}
TuplestupOrdered immutable sequence of objects: (10,”hello”,200.3)
SetssetUnordered collection of unique objects: {“a”,”b”}
BooleansboolLogical value indicating True or False

Variable Assignments

Data Structures

Lists

  • Definition:
    • In Python, a list is a collection of elements, ordered and mutable.
  • Syntax:
    1

    my_list = [1, 2, 3, 'apple', 'banana']
  • Mutability:
    • Lists are mutable, meaning you can change, add, or remove elements after the list is created.
  • Use Cases:
    • Suitable for an ordered collection of items where you might need a flexible data structure.
  • Common Operations:
    • Append (list.append()): Adds an element to the end of the list.
    • Extend (list.extend()): Adds all elements of an iterable to the end of the list.
    • Insert (list.insert()): Inserts an element at a given index.
    • Remove (list.remove()): Removes the first matching element from the list.
    • Pop (list.pop()): Removes and returns the element at a given index.
    • Clear (list.clear()): Removes all elements from the list.
    • Index (list.index()): Returns the index of the first matching element.
    • Count (list.count()): Returns the number of times an element appears in the list.
    • Sort (list.sort()): Sorts the list in place.
    • Reverse (list.reverse()): Reverses the elements of the list in place.
    • Copy (list.copy()): Returns a shallow copy of the list.
    • Membership (element in list): Checks if an element is present in the list.
    • Concatenation (list1 + list2): Combines two lists.
    • Length (len(list)): Returns the number of elements in the list.
    • Iteration (for element in list): Iterates over the elements of the list.
    • Slicing (list[start:end:step]): Extracts a part of the list.
    • Sort (sorted(list)): Returns a sorted list from the given iterable.
    • Max (max(list)): Returns the maximum element of the list.
    • Min (min(list)): Returns the minimum element of the list.
    • Sum (sum(list)): Returns the sum of all elements in the list.
    • Any (any(list)): Checks if any of the elements is True.
    • All (all(list)): Checks if all elements are True.
    • Enumerate (enumerate(list)): Returns an iterator of tuples containing indices and values of the list.
    • Filter (filter(function, list)): Filters elements from the list using a function.
      1
2

      # Using filter and lambda function
positive_numbers = list(filter(lambda x: x > 0, numbers))
    • Map (map(function, list)): Applies a function to every element of the list.
      1
2

      # Using map and lambda function to square each element
squared_numbers = list(map(lambda x: x**2, numbers))
    • Reduce (reduce(function, list)): Applies a rolling computation to the list.
      1
2

      # Using reduce and lambda function to find the product of all elements
product = reduce(lambda x, y: x * y, numbers)

Stack

  • Definition:

    • A stack is a data structure that stores elements in a Last In, First Out (LIFO) manner.

  • Common Operations:

    • Push (stack.append()): Adds an element to the top of the stack.
    • Pop (stack.pop()): Removes and returns the top element of the stack.
    • Peek (stack[-1]): Returns the top element of the stack without removing it.
    • Size (len(stack)): Returns the number of elements in the stack.

    • Empty (not stack): Checks if the stack is empty.

    • Example:
    1
2
3
4
5
6
7
8

    stack = []
stack.append(1)  # [1]
stack.append(2)  # [1, 2]
stack.append(3)  # [1, 2, 3]
stack[-1]        # 3
stack.pop()      # [1, 2]
stack.pop()      # [1]
stack.pop()      # []

  • Use Cases:

    • Useful for implementing algorithms such as depth-first search and backtracking.

Queue

  • Definition:

    • A queue is a data structure that stores elements in a First In, First Out (FIFO) manner.

  • Common Operations:

    • Enqueue (queue.append()): Adds an element to the end of the queue.
    • Dequeue (queue.pop(0)): Removes and returns the first element of the queue.
    • Peek (queue[0]): Returns the first element of the queue without removing it.
    • Size (len(queue)): Returns the number of elements in the queue.
    • Empty (not queue): Checks if the queue is empty.

Arrays

  • Definition:
    • Arrays are a data structure that stores elements of the same type in contiguous memory locations.

  • Syntax:

    • In Python, arrays are often implemented using the NumPy library.
    1
2
3

    import numpy as np

my_array = np.array([1, 2, 3, 4, 5])
  • Mutability:
    • NumPy arrays are mutable like lists but are more efficient for numerical operations.
  • Use Cases:
    • Useful for mathematical operations and large datasets.

Dictionaries

  • Definition:
    • A dictionary is an unordered collection of key-value pairs.
  • Syntax:
    1

    my_dict = {'name': 'John', 'age': 30, 'city': 'New York'}
  • Mutability:
    • Dictionaries are mutable, allowing the addition, modification, and removal of key-value pairs.
  • Use Cases:
    • Ideal for representing real-world entities with attributes, settings, or configurations.

Tuple

A tuple is a data structure in Python that is similar to a list but with one key difference: it is immutable. This means that once you create a tuple, you cannot modify its elements. Tuples are defined using parentheses ().

Here are some key aspects of tuples and where they are commonly used:

  1. Syntax:

    • Tuples are created using parentheses. For example: my_tuple = (1, 2, 3)

  2. Immutability:

    • Once a tuple is created, you cannot change, add, or remove elements from it. This immutability makes tuples useful in situations where you want to ensure that the data remains constant.

  3. Use as Dictionary Keys:

    • Tuples are hashable, which means they can be used as keys in dictionaries. Lists, being mutable, cannot be used as dictionary keys. This hashability is useful in situations where you need to create a compound key for a dictionary.
    1

    my_dict = {('a', 1): 'value', ('b', 2): 'another value'}

  4. Multiple Return Values:

    • Functions in Python can return multiple values as a tuple. When a function returns multiple values, they are packed into a tuple, and you can easily unpack them when calling the function.
    1
2
3
4

    def get_coordinates():
    return 3, 4

x, y = get_coordinates()

  5. Unpacking:

    • Tuples can be used for efficient unpacking of values. This is commonly used in multiple assignment statements.
    1
2

    point = (3, 4)
x, y = point

  6. Ordered Sequences:

    • Tuples, like lists, are ordered sequences. This means the order of elements in a tuple is preserved.
    1

    my_tuple = (1, 2, 3)

  7. Used in zip Function:

    • Tuples are often used in conjunction with the zip function to combine multiple iterables.
    1
2
3
4

    names = ('Alice', 'Bob', 'Charlie')
ages = (25, 30, 35)

combined = zip(names, ages)  # Returns an iterable of tuples

Overall, tuples are a versatile data structure in Python, and their immutability and hashability make them suitable for various use cases, particularly where you need fixed, ordered collections of elements.

Sets

  • Definition:

    • A set is an unordered collection of unique elements in Python.
    • Sets are defined using curly braces {} or the set() constructor.
    1

    my_set = {1, 2, 3, 4, 5}

  • Unique Elements:

    • Sets do not allow duplicate elements. If you try to add a duplicate, it won’t be included in the set.
    1

    my_set = {1, 2, 2, 3, 3, 4}  # Results in {1, 2, 3, 4}

  • Common Set Operations:

      • Union (|): Combines elements from two sets.
      • Intersection (&): Retrieves common elements from two sets.
      • Difference (-): Retrieves elements present in the first set but not in the second.
      • Symmetric Difference (^): Retrieves elements present in either of the sets, but not both.
    1
2
3
4
5
6
7

    set1 = {1, 2, 3}
set2 = {3, 4, 5}

union_set = set1 | set2  # {1, 2, 3, 4, 5}
intersection_set = set1 & set2  # {3}
difference_set = set1 - set2  # {1, 2}
symmetric_difference_set = set1 ^ set2  # {1, 2, 4, 5}

Hashing

  • Definition:

    • Hashing is a process of converting input (or ‘message’) into a fixed-length string of characters, which is typically a hash code.
    • In Python, hash values are used for various purposes, such as indexing elements in dictionaries or sets.

  • Immutable Objects and Hashing:

    • In Python, only immutable objects (objects that cannot be changed after creation) are hashable.
    • Immutable objects like integers, floats, strings, and tuples have fixed values and can be hashed.

  • Hash Function:

    • Python uses a built-in hash function (hash()) to generate hash values for objects.
    1

    hash_value = hash("example")

  • Use in Sets and Dictionaries:

    • Sets and dictionaries use hash values to quickly locate elements.
    • For example, when you check membership in a set or dictionary, Python uses the hash value to determine if the element is present.

  • Custom Hashing:

    • For custom objects or user-defined classes, you can define a __hash__ method to provide a custom hash function.
1
2
3
4
5
6

class CustomObject:
    def __init__(self, value):
        self.value = value

    def __hash__(self):
        return hash(self.value)

Operations

  1. and (Logical AND):

    • It is a logical operator used for combining two or more boolean expressions.
    • It returns True if both operands are true, otherwise, it returns False.
    • Example:
      1

      result = (x > 0) and (y < 10)

  2. & (Bitwise AND):

    • It is a bitwise operator used for performing bitwise AND operation on integers.
    • It compares each bit of the first operand to the corresponding bit of the second operand. If both bits are 1, the result bit is set to 1. Otherwise, it is set to 0.
    • Example:
      1

      result = x & y

and is used for logical operations on boolean values, whereas & is used for bitwise operations on integers. Using and in a bitwise context or & in a logical context might lead to unexpected behavior or errors. It’s important to use the appropriate operator based on the context in which you are working.

##

Item 11: Know How to Slice Sequences

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19

a = ['a', 'b', 'c', 'd', 'e', 'f', 'g', 'h']

# Basic slicing
print('Middle two:', a[3:5])
print('All but ends:', a[1:7])

# Negative indexing
print(a[:5] == a[0:5])
print(a[5:] == a[5:len(a)])

# Various slicing examples
print(a[:])      # Entire list
print(a[:5])     # First five elements
print(a[:-1])    # All but the last element
print(a[4:])     # Elements from index 4 to the end
print(a[-3:])    # Last three elements
print(a[2:5])    # Elements from index 2 to 4
print(a[2:-1])   # Elements from index 2 to the second-to-last
print(a[-3:-1])  # Elements from the third-to-last to the second-to-last

Item 12: Avoid Striding and Slicing in a Single Expression

1
2
3
4
5
6
7
8
9
10
11
12

# Striding examples
x = ['red', 'orange', 'yellow', 'green', 'blue', 'purple']
odds = x[::2]
evens = x[1::2]

# Striding with negative values
x[::-2]  # Reverse every second item
x[2::2]  # Select every second item starting at index 2

# Avoid striding and slicing in a single expression
y = x[::2]  # Good
z = y[1:-1]  # Good

Item 13: Prefer Catch-All Unpacking Over Slicing

1
2
3
4
5
6
7
8
9
10

# Basic unpacking
car_ages = [0, 9, 4, 8, 7, 20, 19, 1, 6, 15]
oldest, second_oldest = car_ages[:2]

# Catch-all unpacking
oldest, second_oldest, *others = car_ages_descending

# Using catch-all unpacking for slices
oldest, *others, youngest = car_ages_descending
*others, second_youngest, youngest = car_ages_descending

Item 14: Sort by Complex Criteria Using the key Parameter

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25

class Tool:
    def __init__(self, name, weight):
        self.name = name
        self.weight = weight

tools = [
    Tool('level', 3.5),
    Tool('hammer', 1.25),
    Tool('screwdriver', 0.5),
    Tool('chisel', 0.25),
]

# Sorting by name
tools.sort(key=lambda x: x.name)

# Sorting by weight
tools.sort(key=lambda x: x.weight)

# Sorting by weight descending, then by name ascending
tools.sort(key=lambda x: (x.weight, x.name))

# Sorting by weight descending, then by name ascending (alternative)
tools.sort(key=lambda x: -x.weight)
tools.sort(key=lambda x: x.name)

Item 15: Be Cautious When Relying on dict Insertion Ordering

  • In Python 3.5 and earlier, iterating over a dict did not guarantee order.
  • Starting with Python 3.6, dictionaries preserve insertion order.
  • Be cautious when relying on insertion order for dict-related operations.
1
2
3
4
5
6
7

# Python 3.5
baby_names = {'cat': 'kitten', 'dog': 'puppy'}
print(list(baby_names.keys()))  # May not preserve insertion order

# Python 3.6+
baby_names = {'cat': 'kitten', 'dog': 'puppy'}
print(list(baby_names.keys()))  # Preserves insertion order

Item 16: Prefer get Over in and KeyError to Handle Missing Dictionary Keys

  • Use get method for cleaner code when handling missing keys.
  • It’s efficient and readable compared to in checks or catching KeyError.
1
2
3
4
5
6

votes = {'otter': 1281, 'polar bear': 587, 'fox': 863}
key = 'wombat'

# Using get
count = votes.get(key, 0)
votes[key] = count + 1

Item 17: Prefer defaultdict Over setdefault to Handle Missing Items in Internal State

  • defaultdict from collections module simplifies handling missing keys.
  • It’s more efficient and avoids unnecessary object creation compared to setdefault.
1
2
3
4
5
6
7
8
9
10
11
12
13

from collections import defaultdict

class Visits:
    def __init__(self):
        self.data = defaultdict(set)

    def add(self, country, city):
        self.data[country].add(city)

# Usage
visits = Visits()
visits.add('France', 'Paris')
print(visits.data)

Item 18: Know How to Construct Key-Dependent Default Values with __missing__

  • Subclass dict and implement __missing__ method to customize default value creation based on keys.
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18

def open_picture(profile_path):
    try:
        return open(profile_path, 'a+b')
    except OSError:
        print(f'Failed to open path {profile_path}')
        raise

class Pictures(dict):
    def __missing__(self, key):
        value = open_picture(key)
        self[key] = value
        return value

# Usage
pictures = Pictures()
handle = pictures['profile_1234.png']
handle.seek(0)
image_data = handle.read()

Item 22: Reduce Visual Noise with Variable Positional Arguments

Accepting a variable number of positional arguments can make a function call clearer and reduce visual noise. These positional arguments are often called varargs or star args.

Example 1: Basic Usage of *args

1
2
3
4
5
6
7
8
9
10

def log(message, *values):
    if not values:
        print(message)
    else:
        values_str = ', '.join(str(x) for x in values)
        print(f'{message}: {values_str}')

# Call the function with multiple values
log('My numbers are', 1, 2)
log('Hi there')  # No values passed

In this example, the *values syntax allows the function to accept a variable number of positional arguments. The function prints the message and, if values are provided, it prints them as well.

Example 2: Using *args with a Sequence

1
2

favorites = [7, 33, 99]
log('Favorite colors', *favorites)

The * operator can be used to pass items from a sequence as positional arguments to a function. This is useful when you have a list of values to pass.

Pitfalls:

  1. Generator Exhaustion:

    1
2
3
4
5
6
7
8
9

    def my_generator():
    for i in range(10):
        yield i

def my_func(*args):
    print(args)

it = my_generator()
my_func(*it)  # This may consume a lot of memory

    Be cautious when using *args with generators, as it can lead to generator exhaustion.

  2. Adding New Positional Arguments:

    1
2
3
4
5
6
7
8
9
10

    def log(sequence, message, *values):
    if not values:
        print(f'{sequence} - {message}')
    else:
        values_str = ', '.join(str(x) for x in values)
        print(f'{sequence} - {message}: {values_str}')

log(1, 'Favorites', 7, 33)        # New with *args OK
log(1, 'Hi there')                 # New message only OK
log('Favorite numbers', 7, 33)    # Old usage breaks

    Adding new positional parameters to functions that accept *args can introduce hard-to-detect bugs. Consider using keyword-only arguments for extension.

Things to Remember:

  • Functions can accept a variable number of positional arguments by using *args in the def statement.
  • The * operator can be used to pass items from a sequence as positional arguments.
  • Be cautious with generators, as using *args with them may cause memory issues.
  • Adding new positional parameters to functions accepting *args can introduce bugs; consider using keyword-only arguments.

Item 23: Provide Optional Behavior with Keyword Arguments

Keyword arguments allow you to provide optional parameters to a function, making it more flexible and self-documenting. This can improve the clarity of function calls and make your code more maintainable.

Example:

1
2
3
4
5
6
7
8
9
10
11
12
13
14

def create_person(name, age, job=None, city=None):
    person_info = f"{name}, {age} years old"
    if job:
        person_info += f", works as a {job}"
    if city:
        person_info += f", lives in {city}"
    return person_info

# Call the function with different sets of arguments
person1 = create_person("Alice", 30, job="Engineer", city="New York")
person2 = create_person("Bob", 25, city="San Francisco")

print(person1)
print(person2)

In this example, the create_person function takes mandatory parameters (name and age) and two optional parameters (job and city). By using keyword arguments, you can choose which optional parameters to include when calling the function.

Benefits of Keyword Arguments:

  1. Clarity and Readability: Keyword arguments make it clear which values correspond to which parameters, improving code readability.

  2. Default Values: Setting default values for keyword arguments allows you to define optional parameters without requiring them in every function call.

  3. Flexibility: Users can choose to provide only the necessary information, ignoring the optional parameters they don’t need.

Pitfall:

  • Mutable Default Values:

    1
2
3
4
5
6
7
8

    def add_item(item, items=[]):
    items.append(item)
    return items

result1 = add_item("apple")
result2 = add_item("banana")

print(result1)  # Output: ['apple', 'banana']

    Be cautious when using mutable objects (like lists) as default values for keyword arguments, as they are shared among all calls to the function.

Things to Remember:

  • Use keyword arguments to provide optional parameters and improve the clarity of function calls.
  • Default values for keyword arguments make parameters optional, and users can choose to override them.
  • Be cautious with mutable default values to avoid unexpected behavior.

Item 24: Use None and Docstrings to Specify Dynamic Default Arguments

When defining a function with default argument values, using mutable objects (like lists or dictionaries) can lead to unexpected behavior due to shared state between calls. Instead, use None as the default and document the behavior clearly using docstrings.

Example:

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22

def add_item(item, items=None):
    """
    Add an item to the list.

    Parameters:
    - item: The item to add.
    - items: The list to which the item will be added. If None, a new list will be created.

    Returns:
    The updated list.
    """
    if items is None:
        items = []
    items.append(item)
    return items

# Test the function
result1 = add_item("apple")
result2 = add_item("banana")

print(result1)  # Output: ['apple']
print(result2)  # Output: ['banana']

In this example, the add_item function uses None as the default value for the items parameter and explicitly checks if it’s None before creating a new list. This avoids the issue of shared state caused by mutable default values.

Benefits:

  1. Predictable Behavior: Using None as the default value ensures that each call to the function starts with a new, independent object.

  2. Clear Documentation: Including a docstring helps users understand the expected behavior of the function, especially regarding default arguments.

Pitfall:

  • Mutable Default Values:

    1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18

    def add_item(item, items=[]):
    """
    Add an item to the list.

    Parameters:
    - item: The item to add.
    - items: The list to which the item will be added. If None, a new list will be created.

    Returns:
    The updated list.
    """
    items.append(item)
    return items

result1 = add_item("apple")
result2 = add_item("banana")

print(result1)  # Output: ['apple', 'banana']

    If you use a mutable object as a default value without handling it properly, you might encounter unexpected behavior due to shared state.

Things to Remember:

  • Use None as the default value for mutable arguments to avoid shared state.
  • Explicitly check for None and create a new instance within the function.
  • Document the function’s behavior, especially regarding default arguments, using docstrings.

1. Explain the difference between lists and tuples in Python.

Lists:

  • Mutable: Can be modified (elements can be added, removed, or changed).
  • Syntax: Defined using square brackets, e.g., my_list = [1, 2, 3].
  • Performance: Slightly slower than tuples due to their mutability.
  • Use Case: When you need a sequence of items that may change.

Tuples:

  • Immutable: Cannot be modified after creation.
  • Syntax: Defined using parentheses, e.g., my_tuple = (1, 2, 3).
  • Performance: Slightly faster than lists due to their immutability.
  • Use Case: When you need a sequence of items that should not change.

2. How do you manage memory in Python?

Memory management in Python involves several aspects:

  • Automatic Garbage Collection: Python uses reference counting and a cyclic garbage collector to automatically manage memory.
    • Reference Counting: Each object maintains a count of references to it. When the reference count drops to zero, the memory is freed.
    • Cyclic Garbage Collector: Handles cyclic references that reference counting alone cannot manage.
  • Memory Pools: Python uses private heap space for object storage. Memory is allocated in pools to reduce fragmentation and improve performance.
  • Memory Profiling Tools: Tools like gc module, pympler, and objgraph can be used to track memory usage and leaks.

Garbage collection in Python refers to the process by which the Python interpreter automatically identifies and frees memory that is no longer in use, thereby making it available for future allocations. This process helps prevent memory leaks and optimizes memory usage in applications.

Key Concepts of Garbage Collection

  1. Reference Counting

    • Python uses reference counting as the primary mechanism to manage memory. Each object has an associated reference count that tracks the number of references pointing to it. When an object’s reference count drops to zero, meaning no references point to it, the memory occupied by the object can be reclaimed.
    1
2
3
4
5
6
7
8
9
10

    import sys

a = []
print(sys.getrefcount(a))  # Output: 2 (a and argument to getrefcount)

b = a
print(sys.getrefcount(a))  # Output: 3 (a, b, and argument to getrefcount)

del b
print(sys.getrefcount(a))  # Output: 2 (a and argument to getrefcount)

  2. Cyclic Garbage Collection

    • Reference counting alone cannot handle cyclic references, where two or more objects reference each other, forming a cycle. Python uses a cyclic garbage collector to detect and collect such cycles.
    1
2
3
4
5
6
7
8
9
10
11
12
13
14
15

    import gc

class Node:
    def __init__(self, value):
        self.value = value
        self.next = None

def create_cycle():
    a = Node(1)
    b = Node(2)
    a.next = b
    b.next = a  # Create a cycle

create_cycle()
gc.collect()  # Force a garbage collection

Practical Example of Garbage Collection

Consider the following example to demonstrate how garbage collection works:

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30

import gc

class Node:
    def __init__(self, value):
        self.value = value
        self.next = None

def create_cycle():
    a = Node(1)
    b = Node(2)
    a.next = b
    b.next = a  # Create a cycle
    return a, b

# Create a cycle
a, b = create_cycle()

# Check the reference counts (they should be > 0)
print("References to 'a':", gc.get_referrers(a))
print("References to 'b':", gc.get_referrers(b))

# Delete references
del a, b

# Force a garbage collection
gc.collect()

# Check again (no references should remain)
print("References to 'a':", gc.get_referrers(a))
print("References to 'b':", gc.get_referrers(b))

How Garbage Collection Works

  1. Reference Counting

    • Each object maintains a count of references to it. When the reference count drops to zero, the memory for that object is immediately reclaimed.

  2. Cyclic Garbage Collection

    • Python’s garbage collector periodically runs to detect cyclic references. The garbage collector uses a three-generation system to track objects: young, middle-aged, and old. Objects that survive garbage collection cycles are promoted to older generations.

  3. Explicit Garbage Collection

    • While Python handles garbage collection automatically, you can manually trigger it using gc.collect(). This is useful for testing and debugging memory issues.

Why Garbage Collection is Important

  • Prevents Memory Leaks: Ensures that memory used by objects that are no longer needed is reclaimed.
  • Optimizes Memory Usage: Frees up memory resources, making them available for other parts of the program.
  • Simplifies Memory Management: Allows developers to focus on the logic of the program without worrying about manual memory allocation and deallocation.

By understanding and utilizing Python’s garbage collection mechanisms, you can write more efficient and memory-safe programs.

3. What are Python decorators and how are they used?

Decorators:

  • Definition: A decorator is a function that takes another function and extends its behavior without explicitly modifying it.
  • Syntax: Defined using the @decorator_name syntax above a function definition.
  • Use Cases: Commonly used for logging, enforcing access control, instrumentation, caching, and more.

Example:

1
2
3
4
5
6
7
8
9
10
11
12

def my_decorator(func):
    def wrapper():
        print("Something is happening before the function is called.")
        func()
        print("Something is happening after the function is called.")
    return wrapper

@my_decorator
def say_hello():
    print("Hello!")

say_hello()

4. How do you handle exceptions in Python?

Exception Handling:

  • Try-Except Block: Used to catch and handle exceptions.
  • Syntax:
    1
2
3
4

    try:
    # Code that may raise an exception
except SomeException as e:
    # Code that runs if the exception occurs
  • Else Block: Executes if no exceptions are raised in the try block.
  • Finally Block: Executes regardless of whether an exception was raised, typically used for cleanup actions.

Example:

1
2
3
4
5
6
7
8

try:
    result = 10 / 0
except ZeroDivisionError as e:
    print(f"An error occurred: {e}")
else:
    print("No errors occurred.")
finally:
    print("This will always execute.")

5. Explain the concept of generators in Python.

Generators:

  • Definition: Generators are a type of iterable, like lists or tuples. Instead of storing all values in memory, they generate values on the fly using the yield keyword.
  • Benefits: Memory efficient, especially for large datasets or infinite sequences.
  • Syntax: Defined like regular functions but use yield instead of return.

Example:

1
2
3
4
5
6
7
8

def my_generator():
    yield 1
    yield 2
    yield 3

gen = my_generator()
for value in gen:
    print(value)

Use Cases: Useful for iterating over large datasets, streaming data, or representing infinite sequences.

In Python, *args and **kwargs are used to allow a function to accept an arbitrary number of arguments.

*args

  • Definition: *args allows a function to accept any number of positional arguments.
  • Usage: Inside the function, args is a tuple containing all the positional arguments passed to the function.

Example:

1
2
3
4
5
6
7
8
9

def example_function(*args):
    for arg in args:
        print(arg)

example_function(1, 2, 3)
# Output:
# 1
# 2
# 3

**kwargs

  • Definition: **kwargs allows a function to accept any number of keyword arguments.
  • Usage: Inside the function, kwargs is a dictionary containing all the keyword arguments passed to the function.

Example:

1
2
3
4
5
6
7
8

def example_function(**kwargs):
    for key, value in kwargs.items():
        print(f"{key}: {value}")

example_function(name="Alice", age=30)
# Output:
# name: Alice
# age: 30

Combined Usage

You can use *args and **kwargs together in a function to accept both positional and keyword arguments.

Example:

1
2
3
4
5
6
7
8
9
10
11
12
13

def example_function(*args, **kwargs):
    for arg in args:
        print(arg)
    for key, value in kwargs.items():
        print(f"{key}: {value}")

example_function(1, 2, 3, name="Alice", age=30)
# Output:
# 1
# 2
# 3
# name: Alice
# age: 30

Application in Decorators

In the context of decorators, *args and **kwargs are often used to pass arguments from the wrapper function to the decorated function.

Example:

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23

def decorator(func):
    def wrapper(*args, **kwargs):
        print("started...")
        result = func(*args, **kwargs)
        print("ended...")
        return result
    return wrapper

@decorator
def func(a, b):
    try:
        print(a // b)
    except Exception as e:
        print(e)
    finally:
        print("Finally")

func(5, 0)
# Output:
# started...
# integer division or modulo by zero
# Finally
# ended...

In this example, *args and **kwargs in the wrapper function ensure that any number of positional and keyword arguments can be passed to the decorated function func.

There are a few issues in the provided code:

  1. The syntax for defining the function parameters is incorrect. In Python, *args must come before **kwargs in function definitions.
  2. The func definition should not have a default parameter (ax="df") before the mandatory parameters (a, b).

Here’s the corrected version of your code:

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21

def decorator(func):
    def wrapper(*args, **kwargs):
        print("started...")
        func(*args, **kwargs)
        print("ended...")
    return wrapper

a = 5
b = 0

@decorator
def func(a, b, ax="df"):
    try:
        print(a // b)
    except Exception as e:
        print(e)
    finally:
        print("Finally")
    return

func(a, b)

Explanation:

  1. Decorator Function:

    • The decorator function defines a wrapper function that accepts *args and **kwargs.
    • The wrapper function prints “started…”, calls the original func with the provided arguments, and then prints “ended…”.
    • The decorator function returns the wrapper function.

  2. Function Definition:

    • The func function is defined to accept a, b, and an optional keyword argument ax with a default value of “df”.
    • Inside func, the division operation and exception handling are performed.

  3. Calling the Decorated Function:

    • The decorated func is called with positional arguments a and b.

Running this corrected code will produce the following output:

1
2
3
4

started...
integer division or modulo by zero
Finally
ended...

This demonstrates the use of *args and **kwargs in both the decorator and the decorated function to handle arbitrary arguments properly.

python
python
This post is licensed under CC BY 4.0 by the author.