Stateless vs stateful code when scaling instances

Many projects run fine with just one instance.

But once you need to scale out to multiple instances, hidden bugs start to appear:

• cron jobs run multiple times
• cache stored in memory becomes inconsistent across machines
• uploaded files exist on one instance but not another
• sessions disappear when the next request lands on a different node

The root cause is often that the code was never designed clearly around stateless vs stateful behavior.

Quick conclusion

If you only remember the short version:

1. stateless design makes horizontal scaling much easier because each instance can handle requests more independently
2. stateful is not automatically wrong, but unmanaged state causes hard-to-debug bugs once you add more instances
3. the most common scaling traps are schedules, local files, local sessions, and data kept in process memory
4. if you want to scale safely, important state should live outside the app process

What does stateless mean?

In practical terms:

• it should not matter whether a request hits instance A or instance B
• the app should not depend on important data living only inside the RAM of one specific instance

Examples that are closer to stateless behavior:

• API requests reading from a database
• auth based on tokens or a shared session store
• files stored in object storage or shared storage
• shared cache stored in Redis

The core idea is:

• the instance runs logic
• but important state does not live exclusively inside that process

What does stateful mean?

Stateful means the app or flow depends on state held inside a specific instance.

Examples:

• user sessions kept in local memory
• runtime maps stored in global variables
• uploaded files written to local disk on one node
• cron jobs on each instance making their own state decisions

These things can still work fine with one machine.
But once you add a second or third instance, problems start.

Why does this matter when scaling?

Because horizontal scaling usually assumes:

• multiple instances can handle requests interchangeably
• any request can land on any instance
• instances can be restarted, replaced, or killed at any time

If your code quietly assumes:

• “this machine holds that data”
• “this cron only runs once”
• “this variable will still be here in memory”

then scaling will expose bugs quickly.

Problem 1: holding important data in process memory

This is a very common mistake.

Example:

const onlineUsers = new Map<string, string>();

If you store business-relevant state like this:

• instance A has its own onlineUsers
• instance B has its own onlineUsers

Once you scale to multiple instances:

• the data is inconsistent
• user state checks become unreliable
• one request hits A and the next request hits B, so the context disappears

If the data matters to the business, it should not live only in the RAM of one process.

A better place is usually:

• Redis
• a database
• a queue backend
• a distributed cache

depending on the use case.

Problem 2: local sessions or auth state

If a session only exists in local memory:

• the user logs in through instance A
• the next request goes to instance B
• the session is gone

That leads to:

• unstable login behavior
• intermittent auth issues
• scaling bugs that look like load balancer problems

If you want reliable multi-instance behavior, sessions should be in:

• Redis
• a database
• or a more stateless token-based approach when appropriate

Problem 3: schedules running multiple times after scaling

This is a very common production pain point.

Examples:

• a cron sends emails
• a cron syncs data
• a cron cleans up old records
• a cron charges billing

With one instance:

• everything looks fine

With three instances:

• the cron runs three times
• duplicate emails go out
• cleanup jobs collide
• billing or sync can be executed multiple times

This is a classic example of hidden stateful behavior inside the app.

If the app has important schedules, you should think in one of these directions:

1. only one dedicated worker runs schedules
2. use a distributed lock
3. use a queue or job system
4. let an external scheduler coordinate execution

In short:

• do not let every web instance run important schedules without coordination

Problem 4: saving files in the source tree or local disk

This is very common in apps that handle uploads or exports.

Example:

• a user uploads an avatar
• the app writes it to ./uploads

With one instance:

• it looks fine

After scaling:

• the file is uploaded to instance A
• the read request later lands on instance B
• the file is missing

And if the container is redeployed or the EC2 instance is replaced:

• the file may disappear entirely

That is why user-uploaded files should not live in:

• the source code folder
• container-local disk
• a path tied to only one instance

A better place is:

• S3
• shared storage
• another object storage service

Problem 5: storing workflow progress inside the app instance

Examples:

• which step a job has reached
• which item is currently being processed
• which batch is still pending

If this state only exists in memory:

• a restart loses it
• scaling makes it inconsistent
• failover becomes harder

Workflow state should usually live in:

• a database
• Redis
• the queue backend

Is stateless always better?

Not everything must become “purely stateless.”

In reality:

• every system has state

The point is not to remove all state.
The real question is:

• where the state lives
• whether it survives restarts
• whether it can be shared between instances
• whether it is durable enough

In other words:

• the business still has state
• but the app instance should not quietly become the only holder of important state

How to apply this so scaling becomes safer

If you want fewer surprises when adding more instances, this checklist is practical:

1. Assume the next request can hit any instance

When writing code, always think:

• the next request may not return to the same machine

That mindset alone prevents a lot of local-state bugs.

2. Move important state outside the app process

Examples:

• sessions -> Redis / DB
• files -> S3 or object storage
• shared cache -> Redis
• workflow state -> DB / queue backend

3. Split web instances from workers or schedulers when needed

This is a very practical design move.

Do not force every web instance to:

• serve requests
• run cron jobs
• process queues

If the system grows, separate the roles:

• web instances
• worker instances
• a dedicated scheduler if needed

4. Make important jobs idempotent

Even if you use locks or queues, important jobs should still be designed so that:

• re-running them does not corrupt data
• a duplicate trigger does not create dangerous side effects

This matters a lot once you start scaling and retrying.

5. Treat local disk as temporary

In horizontally scaled or containerized systems:

• local disk should usually be treated as temporary

Do not store important business data there unless you have a very explicit reason.

A simple way to check whether your code is scaling-friendly

Before increasing the number of instances, ask:

1. if the next request hits another instance, will anything break?
2. if one instance restarts, do I lose important state?
3. if I scale from 1 to 3 instances, will schedules run multiple times?
4. if uploads are local, can another machine read them?
5. if an in-memory variable resets, does business logic break?

If the answer is “yes” in several places, the system is not ready for safe horizontal scaling.

When is stateful still acceptable?

Stateful behavior is not always bad.

Some things can stay local if they are:

• just an optimization
• safe to lose
• not part of business correctness

Examples:

• a small in-memory cache for faster reads
• temporary metrics maps

But it must be very clear that:

• this is an optimization
• not the source of truth

Conclusion

Understanding stateless and stateful is not just architecture vocabulary. It helps you avoid a lot of painful bugs when the system eventually needs more instances.

The most common scaling failures usually involve:

• duplicate schedules
• local file storage
• local sessions
• data held in process memory

If you want safer scaling in the future, one principle matters a lot:

• important state should not be locked inside one app instance

The more stateless your app instances are, the easier horizontal scaling becomes and the fewer surprises you get later.