Get Your Python Configurations Right Every Time with Pydantic Settings

Get Your Python Configurations Right Every Time with Pydantic Settings

Introduction

When you’re developing a serious Python application, you’ll often need configuration parameters that can change based on where the application is running. Environment variables are a great way to handle this because they allow you to adjust settings without changing your application code. Just tweak the environment variables, and you’re good to go — no rebuild required.

However, using environment variables comes with its own set of challenges. You need to ensure that these variables are valid and consistent, or you risk your application failing at runtime. This usually means writing custom code to validate these values, which can be tedious and error-prone. Sometimes, developers skip validation altogether and hope for the best, which is a recipe for disaster.

Enter Pydantic Settings — a fantastic tool that simplifies configuration management in Python applications. With Pydantic Settings, you can harness the power of Pydantic’s data validation to read and validate your environment variables seamlessly. This ensures your application always gets the correct configuration, reducing the risk of runtime errors and making your life as a developer much easier.


Problem Statement

Configuration management is a critical aspect of software development. Applications often require settings for database connections, API keys, and other environment-specific variables. These settings need to be managed securely and efficiently, especially when deploying applications across different environments (development, testing, production).

Hardcoding these settings within your application is not ideal as it can lead to security vulnerabilities and makes it difficult to manage configurations across multiple environments. This is where Pydantic Settings comes into play, offering a structured and secure way to handle configurations.

Examples — With .env

Pydantic Settings makes it easy to load configurations from environment variables or .env files. Let's dive into an example to see how this works.

First, you’ll need to install Pydantic:

pip install pydantic_settings

Next, create a .env file in your project directory with the following content:

DATABASE_URL=postgresql://user:password@localhost:5432/mydatabase
API_KEY=your_api_key_here
DEBUG=True

Now, let’s create a settings.py file to define our settings model using Pydantic:

from pydantic_settings import BaseSettings

class Settings(BaseSettings):
    database_url: str
    api_key: str
    debug: bool

    class Config:
        env_file = ".env"

settings = Settings()

With the Settings class, we define the expected configuration variables: database_urlapi_key, and debug. The Config class nested within Settings specifies that these variables should be loaded from the .env file.

from settings import settings

print(settings.database_url)  # Output: postgresql://user:password@localhost:5432/mydatabase
print(settings.api_key)       # Output: your_api_key_here
print(settings.debug)         # Output: True

If you have a non bool value for debug in your .env file then the above code will throw a validation error.

For example if you change the DEBUG value to xyz in .env file and then re-run the program:

DATABASE_URL=postgresql://user:password@localhost:5432/mydatabase
API_KEY=your_api_key_here
DEBUG=xyz

Then you will get an error like this:

pydantic_core._pydantic_core.ValidationError: 1 validation error for Settings
debug
  Input should be a valid boolean, unable to interpret input [type=bool_parsing, input_value='xyz', input_type=str]
    For further information visit https://errors.pydantic.dev/2.7/v/bool_parsing

So you get data validation for free. You can do any validation with this that you can do with pydantic BaseModel

Nested example

You can also manage your configuration in a nested model. Here’s an example. Notice how we are defining our .env file and how the Settings model is using other models to generate the final configuration. Neat!

# your .env file
V0=0
SUB_MODEL='{"v1": "json-1", "v2": "json-2"}'
SUB_MODEL__V2=nested-2
SUB_MODEL__V3=3
SUB_MODEL__DEEP__V4=v4
from pydantic import BaseModel

from pydantic_settings import BaseSettings, SettingsConfigDict


class DeepSubModel(BaseModel):  
    v4: str


class SubModel(BaseModel):  
    v1: str
    v2: bytes
    v3: int
    deep: DeepSubModel # .env will have __DEEP__ to pass configs to DeepSubModel 


class Settings(BaseSettings):
    v0: str
    sub_model: SubModel
    
    class Config(SettingsConfigDict):
        env_nested_delimiter = "__"


print(Settings().model_dump())
"""
{
    'v0': '0',
    'sub_model': {'v1': 'json-1', 'v2': b'nested-2', 'v3': 3, 'deep': {'v4': 'v4'}},
}
"""

With the nested approach it’s easier to segregate the application configuration and validation logic.

Conclusion

Using Pydantic Settings with nested configurations allows you to organize your application settings more effectively. This structure makes it easier to manage and maintain settings, especially as your application grows. By leveraging the power of Pydantic, you also get type safety and validation out of the box, reducing the risk of configuration errors.

Looking to streamline your Python applications? Feel free to reach out to us at KubeNine Our team specializes in writing and scaling Python applications.

Understand more about Pydantic in our detailed article: https://www.kubeblogs.com/pydantic-fast-and-pythonic-data-validation-for-your-python-applications/