Compare models

Comparing forecast output across models helps you evaluate performance, spot divergence, and build confidence in your predictions. The pd4castr Python SDK lets you fetch comparison data alongside a primary model run in a single call.

Setup

Import the SDK and create a client. All examples on this page assume this setup:

from pd4castr_api_sdk import Client
 
client = Client(
    client_id="your-client-id",
    client_secret="your-client-secret",
)

Compare with a single model

You can pass a comparison_model argument to get_model_run_output to fetch another model’s output alongside the primary run. First, get the latest run for your primary model, then request its output with a comparison:

run = client.get_latest_model_run("primary-model-id")
 
result = client.get_model_run_output(
    "primary-model-id",
    run.id,
    comparison_model="other-model-id",
)
 
print(result.run.model.display_name)
 
# The comparison run is keyed by model ID
comparison = result.comparison_runs["other-model-id"]
print(comparison.model.display_name)

The result.run property contains the primary model’s output. The result.comparison_runs dictionary contains the comparison model’s output, keyed by model ID.

Compare with multiple models

You can compare against several models at once by passing a list of model IDs:

result = client.get_model_run_output(
    "primary-model-id",
    run.id,
    comparison_model=["model-a-id", "model-b-id", "model-c-id"],
)
 
print(f"Comparing against {len(result.comparison_runs)} models")

Each model ID you pass appears as a key in result.comparison_runs.

Access comparison data

Each entry in result.comparison_runs has the same structure as result.run, so you can iterate over comparisons and access their data uniformly:

# Print the primary run
print(f"Primary: {result.run.model.display_name}")
print(f"  Run datetime: {result.run.run_datetime}")
print(f"  Data points: {len(result.run.data)}")
 
# Print each comparison run
for model_id, comparison_run in result.comparison_runs.items():
    print(f"Comparison: {comparison_run.model.display_name}")
    print(f"  Run datetime: {comparison_run.run_datetime}")
    print(f"  Data points: {len(comparison_run.data)}")

Both result.run and each comparison run expose the same fields, including id, run_datetime, model, colours, and data.

Next steps

• Reference — explore every method, data model, and exception in the SDK