Skip to main content
Agentic applications demand near-instant agent availability to maintain user engagement and responsiveness. Pipecat Cloud manages the complexity of scaling these agent deployments in production environments, providing granular controls for compute resources and cost optimization.

​
Core concepts

​
Instances

An instance represents a single unit of compute that runs your agent.
Each instance runs one active session at a time (one bot per instance). Your total concurrency equals the number of running instances, which you control with max-agents. Choosing a larger compute profile (for example agent-2x or agent-3x) gives a single session more CPU and memory; it does not let one instance handle more than one session.
Instance costs are determined by:
  • Active session runtime duration
  • Warm instance maintenance time
  • The compute profile specified in your deployment
Pipecat Cloud automatically provisions and manages agent instances to handle active sessions, ensuring that your deployment can scale to meet demand within the limits of your deployment configuration.

​
Instance pool

Making a deployment to Pipecat Cloud creates a managed pool of agent instances that:
  • Routes requests to available agent instances
  • Scales based on demand within configured limits
  • Maintains optimal performance through auto-scaling
Developers can configure the upper and lower limit of a deployment’s instance pool, providing a cost-effective way to handle varying loads.

​
Minimum agents

min-agents
  • Maintains specified number of warm agent instances to serve incoming requests
  • Immediately ready to become active, reducing cold starts
  • Defaults to 0 if unspecified

Minimum number of agent instances maintained in a pool at all times.

Developers specify a min-agents configuration to determines the number of agent instances that should be kept warm in their deployment pool. A warm instance is kept running and can immediately be used to serve an active session. Maintaining a minimum number of agent instances is important to keep agent start times fast and reduce cold starts.
Setting --min-agents to 1 or greater will incur charges even when the agent is not in use.

​
Maximum agents

max-agents
  • Sets hard limit on concurrent sessions
  • Acts as a cost control / load mechanism
  • Returns HTTP 429 when pool capacity is reached

Maximum agents is the hard limit on the number of agents in your pool.

Each deployment made to Pipecat Cloud has a maximum allowed pool size of 50. Please contact us at help@daily.co or via Discord if you require more capacity.
Deployments can optionally be made with a max-agents configuration that limits the number of agent instances that your pool can contain. This exists as a cost control measure, allowing developers to limit the total number of active sessions that can be run at any one time. The maximum instance count is a hard limit, meaning requests made to a pool that is at capacity will receive a 429 response. See Starting Sessions for more information for how to handle this in your application code.

​
Agent lifecycle

1

Pool Initialization

  • Provisions the minimum number of warm agent instances based on min-agents configuration (defaults to 0)
  • Listens for session requests to route to available agent instances
2

Session Assignment

  • ✅ If a warm instance is available, the session will be assigned to that instance
  • ⏳ If no warm agent instances are available, and your pool is not at capacity, a new instance will be provisioned to handle the request (e.g a cold-start)
  • ❌ If your pool is at capacity, your application will receive a 429 response from the start request
3

Auto-scaling

  • The Pipecat Cloud auto-scaler determines if additional warm agent instances should be created to support further requests.
  • Once a session concludes, the instance is returned to the pool and can immediately serve another session. However, if a new deployment has been pushed since the instance was created, the instance is discarded instead and will not be reused.
Your are billed for warm agent instances, even if they are not handling active sessions. Developers should consider their deployment strategy when cost optimizing, adjusting the minimum and maximum instance count accordingly. See current pricing for details.

​
Cold-starts

A cold start may occur when an active session request is made and no warm agent instances are available in the pool to handle it. In this case, Pipecat Cloud will provision a new instance to handle the request. Cold starts require additional time to provision the instance and load the agent instances, which may result in a delay for the user. To minimize cold starts, you can configure your pool to maintain a minimum number of warm agent instances at all times.
Pipecat Cloud aims to mitigate cold starts as much as possible through auto-scaling.
Around 10 seconds is a best case: a small agent image with an agent that finishes its setup work fast. Treat it as a floor, not a promise.A cold start that runs far longer than 10 seconds, say 30 seconds or a minute or more, usually comes down to your image or your agent’s startup code. Things to check:
  • How big is your image? Every megabyte has to be pulled before your agent can run. Drop what the bot does not need at runtime, and use multi-stage builds to keep build tools out of the final image.
  • What runs at import time in bot.py? Everything at the module level runs before the instance can accept a session. So an import that takes time to load is part of the cold start. It is not only the code inside bot() that costs you time.
  • Are you fetching model files at runtime? Bake model and VAD weights into the image at build time. Then the instance starts with the files already on disk instead of downloading them on the first call.
  • Do you need instant starts? Keep at least one warm instance with min-agents, and plan a waiting experience (a hold message on phone calls, a “connecting” state in web apps) for the times a cold start still happens.
Once you have a running instance of a deployed image, new sessions start immediately with no cold start delay.For developers, especially during test and development, using scale-to-zero with cold starts is a good place to start as it saves money. Once you understand your utilization pattern, you can build a strategy around optimizing start times by maintaining warm agent instances.
To avoid cold starts, you can:
  • Adjust the number of warm agent instances (min-agents) in your pool to ensure that there are always agent instances available to handle requests.
  • Adjust your maximum instance count (max-agents) and issue capacity notifications in your application.

​
Scale-to-zero

For some deployments, using a minimum instance count of 0 is preferable (e.g. while in development.) Since you are only charged for warm agents instances and active sessions, this can be a cost-effective way to manage deployments where fast start times are not required. When the minimum instance count is set to 0, the pool will scale down to 0 agent instances when there are no active sessions. Idle agent instances are maintained for 5 minutes before being terminated.
Scale-to-zero is not recommended for production deployments where immediate response is required.

​
Auto-scaling

Pipecat Cloud performs auto-scaling by default on all deployments. Auto-scaling is accomplished through the following mechanisms:
  • Scaling up based on request velocity
  • Maintaining efficiency within max-agents limit
  • Scaling down to min-agents (or zero) during low usage
  • Supporting burst workloads automatically

​
Auto-Scaling Buffer

Pipecat Cloud maintains a free auto-scaling buffer in addition to your paid reserved agent instances. This saves you from over-provisioning warm agent instances while still ensuring fast response times during traffic increases.
When your traffic increases and you have active sessions running, our system automatically:
  • Proactively provisions additional idle agent instances based on your current usage patterns
  • Provides these buffer agent instances at no additional cost to you
  • Ensures you can continue handling traffic spikes even when all your paid warm agents are in use
For example, with multiple active sessions:
  • The system is already spinning up additional buffer agent instances in the background
  • These buffer agent instances become available within ~30 seconds
  • You can continue calling the /start endpoint without worrying about configuring additional capacity
This approach means:
  • You can set a lower min-agents value than your peak traffic requirements
  • You’ll still avoid cold starts in most scenarios
  • You get better cost efficiency without sacrificing performance
The only scenario where you need to consider higher min-agents values is for extremely rapid traffic spikes (tens or hundreds of calls per second) where the buffer can’t be provisioned fast enough.

​
Scaling Philosophy

Our scaling system is designed to minimize the need for manual capacity planning. Here’s how we recommend thinking about scaling:
  • Start simple: We encourage you to set min-agents to 0 initially and test how the system performs for your specific use case. Many applications work well without any pre-warmed agent instances.
  • Optimize as needed: We work hard to make cold starts rare and as fast as possible so that, for many applications, you don’t have to worry about warm instances at all.
  • Tune for traffic patterns: If you have spiky workloads with bursty traffic patterns, setting an appropriate min-agents value can help prevent cold starts during critical periods. Consider scheduling higher min-agents values only during your peak usage hours.
Our goal is to make scaling decisions as simple as possible while giving you the controls you need for optimization when required.

​
Updating scaling configuration

You can update your deployment’s configuration at any time via the CLI or Pipecat Cloud Dashboard.
Please note that changing your scaling parameters will not disrupt any active sessions. If you reduce your max instance count below the number of currently active sessions, you will still be billed for the duration of those sessions. A scaling-only change also doesn’t create a new version or restart your instances, so it won’t pick up new code, a new image, or a new secret value (why).

​
Capacity Planning

Effective capacity planning is crucial for production deployments to ensure your agents respond immediately. Pipecat Cloud auto-scales your agents. For most cases, the only action you need to take is to set the --min-agents parameter to 1. However, if your application experiences fluctuations in traffic, you may need to plan for additional warm capacity to ensure your agents are always ready to respond immediately.

Capacity Planning Guide

See our guide for calculating reserved agents, understanding warm capacity, and implementing scaling strategies for production.

​
Usage summary

Pipecat Cloud bills based on:
  • Active session minutes: Time your agents spend handling live sessions
  • Reserved session minutes: Time your warm agent instances are kept running, even when idle
An active session starts when you call the /start endpoint (or CLI or SDK equivalent) and ends when your agent’s pipeline shuts down. Reserved session minutes are optional and controlled by setting --min-agents in your deployment configuration.
Both active and reserved session time is measured to the second and billed in minutes.

​
Controlling costs

Most surprise charges come from reserved session minutes: a warm instance bills around the clock whether or not anyone uses it. The levers below are ordered from most to least impactful.

​
Scale-to-zero in development

If you don’t need instant start times while testing, set min-agents to 0 to remove all reserved charges. The first call after an idle period pays a cold start: around 10 seconds at best, and longer for a large image or an agent that does heavy setup work. Later calls are instant while an instance stays warm.
Each warm instance bills around the clock. A single instance kept warm with --min-agents 1 accrues 1,440 reserved minutes a day, even with zero active sessions.

​
Right-size min-agents in production

For production, set min-agents to cover your baseline traffic and let the free auto-scaling buffer handle spikes. See Capacity Planning for how to pick the number.

​
Cap concurrency with max-agents

max-agents sets a hard limit on how many instances, and therefore concurrent sessions, your pool can run. It doubles as a cost ceiling: requests past the limit get a 429 instead of spinning up more paid instances.

​
Cap runaway sessions with max-session-duration

A buggy agent that never ends its pipeline can rack up active minutes. Set --max-session-duration as a safety net so any session is force-closed after a set time.
See session duration limits for the default and allowed range.

​
Delete deployments you are not using

A deployment with warm instances accrues reserved charges until you delete it. If you’re done with an app, delete the deployment.

​
Set a spend limit as a backstop

As a final safety net, set a spend limit in the Pipecat Cloud Dashboard under Settings > Billing. It caps the damage from a misconfiguration but doesn’t replace right-sizing min-agents.