Skip to main content

💸 Spend Tracking

Track spend for keys, users, and teams across 100+ LLMs.

Getting Spend Reports - To Charge Other Teams, API Keys

Use the /global/spend/report endpoint to get daily spend per team, with a breakdown of spend per API Key, Model

Example Request

curl -X GET 'http://localhost:4000/global/spend/report?start_date=2024-04-01&end_date=2024-06-30' \
-H 'Authorization: Bearer sk-1234'

Example Response

[
{
"group_by_day": "2024-04-30T00:00:00+00:00",
"teams": [
{
"team_name": "Prod Team",
"total_spend": 0.0015265,
"metadata": [ # see the spend by unique(key + model)
{
"model": "gpt-4",
"spend": 0.00123,
"total_tokens": 28,
"api_key": "88dc28.." # the hashed api key
},
{
"model": "gpt-4",
"spend": 0.00123,
"total_tokens": 28,
"api_key": "a73dc2.." # the hashed api key
},
{
"model": "chatgpt-v-2",
"spend": 0.000214,
"total_tokens": 122,
"api_key": "898c28.." # the hashed api key
},
{
"model": "gpt-3.5-turbo",
"spend": 0.0000825,
"total_tokens": 85,
"api_key": "84dc28.." # the hashed api key
}
]
}
]
}
]

Allowing Non-Proxy Admins to access /spend endpoints

Use this when you want non-proxy admins to access /spend endpoints

Create Key

Create Key with with permissions={"get_spend_routes": true}

curl --location 'http://0.0.0.0:4000/key/generate' \
--header 'Authorization: Bearer sk-1234' \
--header 'Content-Type: application/json' \
--data '{
"permissions": {"get_spend_routes": true}
}'

Use generated key on /spend endpoints

Access spend Routes with newly generate keys

curl -X GET 'http://localhost:4000/global/spend/report?start_date=2024-04-01&end_date=2024-06-30' \
-H 'Authorization: Bearer sk-H16BKvrSNConSsBYLGc_7A'

Reset Team, API Key Spend - MASTER KEY ONLY

Use /global/spend/reset if you want to:

  • Reset the Spend for all API Keys, Teams. The spend for ALL Teams and Keys in LiteLLM_TeamTable and LiteLLM_VerificationToken will be set to spend=0

  • LiteLLM will maintain all the logs in LiteLLMSpendLogs for Auditing Purposes

Request

Only the LITELLM_MASTER_KEY you set can access this route

curl -X POST \
'http://localhost:4000/global/spend/reset' \
-H 'Authorization: Bearer sk-1234' \
-H 'Content-Type: application/json'

Expected Responses

{"message":"Spend for all API Keys and Teams reset successfully","status":"success"}

Spend Tracking for Azure

Set base model for cost tracking azure image-gen call

Image Generation

model_list: 
- model_name: dall-e-3
litellm_params:
model: azure/dall-e-3-test
api_version: 2023-06-01-preview
api_base: https://openai-gpt-4-test-v-1.openai.azure.com/
api_key: os.environ/AZURE_API_KEY
base_model: dall-e-3 # 👈 set dall-e-3 as base model
model_info:
mode: image_generation

Chat Completions / Embeddings

Problem: Azure returns gpt-4 in the response when azure/gpt-4-1106-preview is used. This leads to inaccurate cost tracking

Solution ✅ : Set base_model on your config so litellm uses the correct model for calculating azure cost

Get the base model name from here

Example config with base_model

model_list:
- model_name: azure-gpt-3.5
litellm_params:
model: azure/chatgpt-v-2
api_base: os.environ/AZURE_API_BASE
api_key: os.environ/AZURE_API_KEY
api_version: "2023-07-01-preview"
model_info:
base_model: azure/gpt-4-1106-preview