pydantic_ai.usage
Bases: UsageBase
LLM usage associated with a single request.
This is an implementation of genai_prices.types.AbstractUsage so it can be used to calculate the price of the
request using genai-prices.
def __add__(other: RequestUsage) -> RequestUsage
Add two RequestUsages together.
This is provided so it’s trivial to sum usage information from multiple parts of a response.
WARNING: this CANNOT be used to sum multiple requests without breaking some pricing calculations.
@classmethod
def extract(
cls,
data: Any,
*,
provider: str,
provider_url: str,
provider_fallback: str,
api_flavor: str = 'default',
details: dict[str, Any] | None = None,
) -> RequestUsage
Extract usage information from the response data using genai-prices.
data : Any
The response data from the model API.
provider : str
The actual provider ID
provider_url : str
The provider base_url
provider_fallback : str
The fallback provider ID to use if the actual provider is not found in genai-prices. For example, an OpenAI model should set this to “openai” in case it has an obscure provider ID.
api_flavor : str Default: 'default'
The API flavor to use when extracting usage information, e.g. ‘chat’ or ‘responses’ for OpenAI.
Becomes the details field on the returned RequestUsage for convenience.
def incr(incr_usage: RequestUsage) -> None
Increment the usage in place.
incr_usage : RequestUsage
The usage to increment by.
Bases: UsageBase
LLM usage associated with an agent run.
Responsibility for calculating request usage is on the model; Pydantic AI simply sums the usage information across requests.
Total number of audio tokens read from the cache.
Type: int Default: 0
Total number of tokens read from the cache.
Type: int Default: 0
Total number of tokens written to the cache.
Type: int Default: 0
Any extra details returned by the model.
Type: dict[str, int] Default: dataclasses.field(default_factory=(dict[str, int]))
Total number of audio input tokens.
Type: int Default: 0
Total number of input/prompt tokens.
Type: int Default: 0
Total number of output/completion tokens.
Type: int Default: 0
Number of requests made to the LLM API.
Type: int Default: 0
Number of successful tool calls executed during the run.
Type: int Default: 0
def __add__(other: RunUsage | RequestUsage) -> RunUsage
Add two RunUsages together.
This is provided so it’s trivial to sum usage information from multiple runs.
def incr(incr_usage: RunUsage | RequestUsage) -> None
Increment the usage in place.
incr_usage : RunUsage | RequestUsage
The usage to increment by.
Number of audio tokens read from the cache. Included in cache_read_tokens and input_audio_tokens.
Type: int Default: 0
Fraction of input tokens that were read from the provider’s prompt cache.
Computed as cache_read_tokens / input_tokens. Both counts span all modalities — cached audio tokens are
included in cache_read_tokens just as audio input tokens are included in input_tokens — and
input_tokens includes cached reads for every provider, so the ratio is comparable across providers:
0.0 means no prompt-cache hits, while values approaching 1.0 mean nearly the entire prompt was served
from cache. Returns 0.0 when there are no input tokens.
On RequestUsage this is the hit ratio of a single request; on
RunUsage it aggregates all requests in the run.
Type: float
Number of tokens read from the cache, across all modalities (includes cache_audio_read_tokens).
Included in input_tokens.
Type: int Default: 0
Number of tokens written to the cache. Included in input_tokens.
Type: int Default: 0
Best-effort cost in USD, or None if no cost could be determined.
Calculated with genai-prices. None (rather than zero) when the
model or provider can’t be priced, so “unknown” stays distinguishable from a genuine zero cost.
Type: Decimal | None Default: None
Any extra details returned by the model.
Type: Annotated[dict[str, int], BeforeValidator(lambda d: d or {})] Default: details or {}
Number of audio input tokens. Included in input_tokens.
Type: int Default: 0
Total number of input/prompt tokens, across all modalities.
Token counts form inclusive parent/child buckets, not disjoint ones: this total includes cached
tokens (cache_read_tokens, cache_write_tokens) and audio tokens (input_audio_tokens).
Usage extraction normalizes providers that report these separately (e.g. Anthropic and Bedrock,
whose raw input_tokens exclude cache reads/writes) so the convention holds everywhere.
Type: Annotated[int, Field(validation_alias=(AliasChoices(input_tokens, request_tokens)))] Default: 0
Number of audio output tokens. Included in output_tokens.
Type: int Default: 0
Number of output/completion tokens.
Type: Annotated[int, Field(validation_alias=(AliasChoices(output_tokens, response_tokens)))] Default: 0
Sum of input_tokens + output_tokens.
Type: int
def __copy__() -> UsageBase
Shallow copy that also copies mutable fields like details.
UsageBase
@classmethod
def __get_pydantic_core_schema__(
cls,
source_type: Any,
handler: GetCoreSchemaHandler,
) -> core_schema.CoreSchema
Preserve arbitrary usage fields across Pydantic serialization.
core_schema.CoreSchema
def has_values() -> bool
Whether any values are set and non-zero.
def opentelemetry_attributes() -> dict[str, int]
Get the token usage values as OpenTelemetry attributes.
Limits on model usage.
The request count is tracked by pydantic_ai, and the request limit is checked before each request to the model. Token counts are provided in responses from the model, and the token limits are checked after each response.
Each of the limits can be set to None to disable that limit.
The maximum cost allowed in USD.
Type: Decimal | None Default: None
If True, perform a token counting pass before sending the request to the model,
to enforce input_tokens_limit and per_request_input_tokens_limit ahead of time.
This may incur additional overhead (from calling the model’s count_tokens API before making the actual request)
and is disabled by default.
Supported by:
- Anthropic
- Bedrock Converse
- OpenAI Responses
Type: bool Default: False
The maximum number of input/prompt tokens allowed.
Type: int | None Default: None
The maximum number of output/response tokens allowed.
Type: int | None Default: None
The maximum number of input/prompt tokens allowed per individual request.
Unlike input_tokens_limit which is cumulative across the entire run, this
limit is checked against each request’s input token count independently —
ahead of the request when count_tokens_before_request=True, otherwise against
the provider-reported input_tokens of the response.
This provides a guard against oversized contexts (which hurt model performance and incur high costs on cache misses), complementing the runaway-loop protection that cumulative limits provide.
Note that input_tokens (and therefore this limit) includes cached-prefix tokens,
normalized consistently across providers: a request served largely from cache still
counts its full context size toward this limit. This caps context size, not cache-miss cost.
Set count_tokens_before_request=True to enforce this preemptively; otherwise the
request is sent before the limit is checked, so the oversized request is still
billed (matching input_tokens_limit).
Type: int | None Default: None
The maximum number of requests allowed to the model.
The maximum number of successful tool calls allowed to be executed.
Type: int | None Default: None
The maximum number of tokens allowed in requests and responses combined.
Type: int | None Default: None
def check_before_request(usage: RunUsage) -> None
Raises a UsageLimitExceeded exception if the next request would exceed any of the limits.
def check_before_tool_call(projected_usage: RunUsage) -> None
Raises a UsageLimitExceeded exception if the next tool call(s) would exceed the tool call limit.
def check_cost(usage: RunUsage, *, warn_if_cost_unavailable: bool = True) -> None
Check whether usage exceeds the cost limit.
usage : RunUsage
The accumulated run usage to check.
Whether to warn when a cost_limit is set but no cost was calculated.
def check_per_request_input_tokens(request_input_tokens: int) -> None
Raises a UsageLimitExceeded if the per-request input tokens exceed the limit.
This checks a single request’s input token count — not the cumulative
RunUsage.input_tokens — against per_request_input_tokens_limit.
def check_tokens(usage: RunUsage) -> None
Raises a UsageLimitExceeded exception if the usage exceeds any of the token limits.
def has_token_limits() -> bool
Returns True if this instance places any limits on token counts.
If this returns False, the check_tokens and check_per_request_input_tokens methods will never raise an error.
This is useful because if we have token limits, we need to check them after receiving each streamed message. If there are no limits, we can skip that processing in the streaming response iterator.