EasyPost’s API Scalability Guide
by EasyPost
EasyPost's infrastructure engineering team has had one key focus in mind over the last few years: API scalability. That hard work has paid off. Our APIs are built on infrastructure ready to support exponential growth and to get our customers through peak season without even a hiccup. This guide delves into API scalability and what innovative methods we employ to offer the most scalable APIs in the industry.
What is API scalability and why is it so important?
API scalability refers to the ability to support concurrent requests without impact on performance.
The ecommerce industry has been exponentially growing year over year since 2015, saw an even higher spike in 2020, and has been growing rapidly ever since. However, many top shipping API providers are built on outdated infrastructure that wasn't designed for the scale we are now seeing as well as the growth we expect to see moving forward.
A truly scalable API supports heavy volume across its entire customer base through the peak shipping season with no impact on performance. Shipping API companies shouldn't be able to hide behind the excuse that shipping API performance decline is unavoidable during peak season. Ecommerce companies invest heavily to meet peak season demand, ramping up inventory, staffing, additional hours, and more. Their shipping API solution should rise to the occasion as well.
Our APIs are designed to support far greater volumes than we see today. We’re constantly improving our infrastructure so we are ready to grow exponentially in the next year, and we proudly publish the performance of our shipping API response time metrics for the last 24 hours here.
The next section outlines how we measure scalability and what customers can expect from our APIs.
How do we measure scalabilty?
We evaluate the scalability of our APIs by measuring the maximum request volume they can handle per second without performance degradation. This helps us ensure that we can onboard new customers and handle seasonal spikes without introducing latency.
Scalability isn’t just about surviving peak season—it’s about maintaining performance when it matters most. Our APIs are designed to stay fast and stable, even under heavy load.
Next, we’ll walk through some of the key innovations driving our scalability today—and what we’re building for the future.
How do we do it?
The following list of items details some of the key innovative methods we employ to optimize API scalability. However, we also employ many common industry best practices not described below.
1. Stateless
We design our APIs to be stateless where possible to optimize request efficiency and reduce server load. Stateless APIs do not rely on prior interaction history because each request contains all the information needed for fulfillment. This architecture allows multiple servers to support requests interchangeably, improving scalability and reliability.
One great example is our use of stateless authentication, where each request is independently verified by using an API key. In contrast, a stateful API relies on information from previous client sessions to fulfill requests. To scale a stateful API across multiple servers, each client session would need to be stored across all of those servers. In cases where we do store information, we regularly analyze the usefulness of that information to ensure we are making the optimal scalability trade-offs.
2. Asynchronous job processing
EasyPost uses asynchronous job processing to rapidly complete bulk operations and prevent backlogs from building up in the system. If two or more jobs do not depend on one another, we run them in parallel (asynchronously) to improve performance. We have seen asynchronous job processing improve performance by up to 10x speed for bulk operations such as purchasing a batch of 1,000 shipments. This is especially critical during the peak shipping season when bulk requests are more common and backlog queues are longer.
3. Hybrid NoSQL and SQL data management approach
EasyPost leverages both SQL and NoSQL databases for much more efficient and scalable data management. Simpler, faster read operations are usually better suited for SQL, whereas heavier write operations—often the bottlenecks for scaling—are better handled by NoSQL databases. For any given operation, we dynamically use whichever infrastructure delivers the best performance and scalability. Our system is set up to scale exponentially with the growth of the ecommerce industry.
4. Regular monitoring and alerting
We reduce problematic bottlenecks that may cause latency for customers by regularly monitoring the load on our system and alerting internal teams if unhealthy thresholds are exceeded. When alerted, we immediately begin work to build and deploy solutions that will return our systems to a healthy state. We have staff on call at all times of the day, ready to respond to critical alerts for all supported systems. Regular monitoring and alerting also gives us an accurate pulse of where our current scalability limitations are, so we can better prioritize builds and push scalability boundaries in the future.
5. Dynamic rate limiting
We use dynamic rate-limiting thresholds to increase max allowable throughput for customers. Different requests cause different loads to our system depending on the amount of information requested. For this reason, we dynamically set thresholds for our rate limits based on the cumulative load on the API rather than a threshold based on the number of requests made to the API. This means users won't be rate limited due to sheer volume when making numerous performant requests in a short span.
Related guides
We know many components comprise an enterprise-grade API beyond just performance. This guide is one part of a series of guides that describe all of the different aspects of our enterprise shipping APIs. Click below to learn more!
Ready to talk with a shipping expert?
Effective API documentation is critical to our customers’ success, and we’re committed to going beyond "good enough." Expect to see our success metrics continually improving as we explore new, innovative ways to support developers at every step.