Cost Estimation for Agent Actions

AISentinel provides a comprehensive cost estimation system to help organizations monitor, control, and optimize the costs associated with agent actions. This system enables proactive cost management through rule-based controls and provides transparency into resource usage.

Overview

The cost estimation system works at multiple levels:

Preflight Estimation: Before executing any agent action, the system estimates the cost
Rule-Based Controls: Organizations can define rules that block or flag expensive actions
Post-Execution Tracking: Actual costs are tracked and can be compared to estimates
Tenant-Level Configuration: Each tenant can customize cost parameters

How Cost Estimation Works

Estimation Algorithm

The system uses a heuristic USD-equivalent cost estimator that combines:

1. Token-Based LLM Costs

Token Estimation: Characters ÷ 4 ≈ tokens (rough approximation)
Per-Token Rate: Configurable USD cost per token
Default Rate: $0.0001 per token (~$0.10 per 1K tokens)

2. Tool-Specific Base Costs

Fixed Costs: Added per tool type regardless of input size
Configurable: Different costs for different tools
Examples:
- Web requests: $0.001
- Python execution: $0.01
- Calculator: $0.00

3. Total Cost Formula

Total Cost = (Token Estimate × Per-Token Rate) + Tool Base Cost

Configuration Options

ESTIMATOR_PER_TOKEN_USD

Type: Float (USD)
Default: 0.0001
Description: Cost per token in USD
Example: 0.00015 (for $0.15 per 1K tokens)

ESTIMATOR_TOOL_BASE_COSTS

Type: JSON string mapping tool names to base costs
Default: {"web": 0.001, "python_exec": 0.01, "calc": 0.0}
Description: Fixed costs added per tool usage
Format: {"tool_name": cost_in_usd, ...}

Configuration Methods

Via Dashboard

Navigate to Settings → Configuration
Find Cost Estimation section
Update Per-Token USD Rate and Tool Base Costs

Via API (Admin Only)

# Set per-token rate
curl -X PUT "/api/v1/tenant-config/ESTIMATOR_PER_TOKEN_USD" \
  -H "Authorization: Bearer $ADMIN_TOKEN" \
  -H "X-AISentinel-Tenant: $TENANT_ID" \
  -d '"0.00015"'

# Set tool base costs
curl -X PUT "/api/v1/tenant-config/ESTIMATOR_TOOL_BASE_COSTS" \
  -H "Authorization: Bearer $ADMIN_TOKEN" \
  -H "X-AISentinel-Tenant: $TENANT_ID" \
  -d '{"web": 0.002, "python_exec": 0.02, "search": 0.005}'

Using Cost Estimates in Rules

Basic Cost Thresholds

# Block expensive actions
when: 'estimate.cost > 5.0'
action: block
message: "Action cost exceeds $5.00 threshold"

# Flag high-cost actions for review
when: 'estimate.cost > 1.0'
action: flag
severity: medium

Complex Cost Rules

# Different thresholds by tool
when: 'tool equals web AND estimate.cost > 0.1'
action: block
message: "Web requests over $0.10 not allowed"

# Cost + risk combination
when: 'estimate.cost > 2.0 OR estimate.risk > 0.8'
action: require_approval
message: "High cost or risk requires approval"

# Budget-based controls
when: 'estimate.cost > 10.0 AND tool in python_exec,web'
action: block
message: "Compute-intensive operations over $10 blocked"

Cost Estimation Examples

Example 1: Web Request

Input: URL fetch with 1000-character response
Token Estimate: 1000 ÷ 4 = 250 tokens
Tool Base Cost: $0.001
Per-Token Cost: 250 × $0.0001 = $0.025
Total Estimate: $0.001 + $0.025 = $0.026

Example 2: Python Execution

Input: 2000-character Python code
Token Estimate: 2000 ÷ 4 = 500 tokens
Tool Base Cost: $0.01
Per-Token Cost: 500 × $0.0001 = $0.05
Total Estimate: $0.01 + $0.05 = $0.06

Example 3: Calculator

Input: Simple math expression
Token Estimate: 50 ÷ 4 = 13 tokens
Tool Base Cost: $0.00
Per-Token Cost: 13 × $0.0001 = $0.0013
Total Estimate: $0.00 + $0.0013 = $0.0013

Best Practices

Setting Cost Thresholds

Conservative Approach

# Start with low thresholds, increase gradually
when: 'estimate.cost > 0.1'
action: flag
severity: low

when: 'estimate.cost > 1.0'
action: require_approval

when: 'estimate.cost > 10.0'
action: block

Tool-Specific Limits

# Web requests - lower tolerance
when: 'tool equals web AND estimate.cost > 0.05'
action: block

# Python execution - higher tolerance
when: 'tool equals python_exec AND estimate.cost > 2.0'
action: require_approval

# Search operations - medium tolerance
when: 'tool equals search AND estimate.cost > 0.5'
action: flag

Monitoring and Adjustment

Track Actual vs Estimated Costs
- Monitor billing data against estimates
- Adjust rates based on actual usage patterns
Gradual Threshold Increases
- Start conservative, increase based on usage patterns
- Use approval workflows for learning period
Regular Reviews
- Monthly review of cost patterns
- Adjust thresholds based on business needs

Troubleshooting

Common Issues

Costs Too High

Check: ESTIMATOR_PER_TOKEN_USD setting
Solution: Reduce per-token rate or adjust algorithm assumptions

Costs Too Low

Check: Tool base costs configuration
Solution: Increase ESTIMATOR_TOOL_BASE_COSTS values

Inaccurate Estimates

Check: Token estimation algorithm (characters ÷ 4)
Solution: Custom token counting for specific use cases

Debugging Cost Estimates

// Check current configuration via dashboard
// Settings → Configuration → Cost Estimation

// View estimates in audit logs via API
fetch('/api/v1/audit?filter=estimate.cost>1.0')

Integration with Billing

Cost Tracking

Estimates are included in audit logs
Actual costs can be correlated with estimates
Billing systems can use estimates for budgeting

Budget Controls

# Daily budget limits
when: 'daily_cost_total + estimate.cost > 100.0'
action: block
message: "Daily budget limit would be exceeded"

# Project-specific budgets
when: 'project_budget_remaining < estimate.cost'
action: require_approval
message: "Insufficient project budget"

This cost estimation system provides organizations with the controls and visibility needed to manage AI agent costs effectively while maintaining operational flexibility.