Thread-Safe Client Libraries Are Here!
by Nate Harris
Eighteen months ago, EasyPost established a dedicated Developer Experience team to refocus our efforts to deliver the best-in-quality experience for software engineers integrating with our shipping solution API. This included giving some much-needed love and attention to our client libraries. Now, after months of hard work, we’d like to highlight all the progress our team has made. Specifically, we’d like to announce and celebrate a significant milestone: all of our client libraries are now thread-safe.
For years, we’ve received feedback from our user base about the difficulty of using our libraries in their applications due to the lack of thread safety, and tackling this has been one of the major goals of the Developer Experience team during our revitalization efforts.
What is thread safety?
For those who are unfamiliar with the term, thread safety is ensuring that a piece of code can run in a multi-threaded environment without threads interfering with one another. Several of our users run our client libraries in their multi-threaded applications, interacting with several different API calls and contexts simultaneously. Unfortunately, our client libraries were not originally built with this workflow in mind.
Specifically, in most of our client libraries, we had some form of either a global API key variable or a singleton HTTP client. If an application ever needed to switch between different API keys to support multiple users at the same time, it could simply alter this global variable or reset the HTTP client. But occasionally, in situations where there were multiple threads making API calls, each with different API keys, these global variables and singletons were being set and reset back and forth as each thread was attempting to execute. Because the timings of these threads aren’t always consistent, it was possible that API keys could get used by the wrong API calls accidentally. This could lead to, for example, one user buying another user’s labels.
Here’s a visualization. Thread 1 needs to set the global API key to Key 1, then make an API call that, under the hood, uses whatever value the global API key is set to. Thread 2 needs to do the same thing, but with Key 2. When running simultaneously, the following is possible:
- Thread 1 sets the global API key to Key 1.
- Thread 2 sets the global API key to Key 2.
- Thread 1 initiates an API call, which uses the value of the global API key.
- Oh no, Thread 1 has just made its API call, but with Thread 2’s API key!
How we fixed this
At first, we experimented with various ways to introduce thread safety into our client libraries with as few architectural changes as possible. But as we got further into our work on the libraries, and had already pushed out multiple major version bumps for some of them, we realized that the best way forward was to lean into an architectural change that would allow us to holistically address the thread safety issues.
Those using our .NET library may remember our first go at adding thread safety back in September with version 4.0.0. Based on our internal metrics (and an eager .NET developer on our team), we determined that the library was a suitable candidate to test our new thread-safe design (Sidenote: Our Go client library, which was written several years after all our other libraries, was designed since its inception to be thread-safe). After receiving feedback on the design and seeing it in action, we iterated further. We did away with the half-and-half approach of the .NET library, which had a combination of “static” functions called against services (categories) and “instance” functions called against objects, and instead moved forward with an all-”static” approach.
What’s the difference?
Previously, most of the API-calling methods were static functions, meaning they weren’t called against an instance of any model or object, and used global variables, such as a global API key and a singleton HTTP client, to make API calls.
Now, all our libraries revolve around an instance of a client, which is configured with its own API key and additional settings. Any HTTP request made via any function on this client uses the client’s API key. Developers using our library can spawn as many instances of a client as they’d like, each with its own API key. The client instances are effectively siloed from one another; no resources are shared between them, so any modification made to one client will not influence another. Most importantly, multiple clients will not be stepping on each other’s toes if they’re both running simultaneously.
Because each HTTP call needs access to a specific client’s API key and settings, rather than global settings, every API-calling function now needs to be attached to an instance of a client.
# Old method
Address myAddress = Address.CreateAddress(params);
# New method
Client myClient = new Client();
Address myAddress = myClient.Address.Create(params);To make our libraries a bit more organized, we grouped methods into appropriate sub-categories we call “services.”
# Address-related functionality available under the `Address` service on the client
Address myAddress = myClient.Address.Create(params)
# Shipment-related functionality available under the `Shipment` service on the client
Shipment myShipment = myClient.Shipment.Buy(id, params)In our first attempt at thread safety in the .NET library, we would store the client used to make or retrieve a given object inside the object itself, reusing it for future API calls. For example, creating an address would return an Address object with the client used to create it available inside. This allowed instance-based, API-calling methods to exist on the Address object that could be used to do things like update the address.
However, when we began porting this functionality to the other client libraries, we realized the shortcomings of this approach, both from a coding perspective (particularly in statically-typed languages like Java) and from a usability perspective. So we scrapped the idea and decided that, instead of calling .Buy() on a Shipment instance, users would call .Buy() on the Shipment service.
# Old method (.NET v4.0.0)
myShipment.Buy(rate);
# New method
myShipment = myClient.Shipment.Buy(myShipment.Id, rate)For the majority of API calls made to EasyPost, when referring to a specific record server-side, such as an address or shipment, the ID is the only thing needed server-side to locate and interact with the record. As a result, the majority of the methods in our libraries only need the ID of the object being modified as a parameter (along with whatever data is being modified on the record).
The second big change in this re-architecture is to the return values of update requests — specifically, they exist now! Previously, the instance of an object being modified would be updated in-place behind the scenes with whatever new data was returned by the API call. This relied on fragile merge functions that often made assumptions about what new data to keep and what to ignore, and it particularly did not play well in our statically-typed languages like Java and .NET.
Since we have removed “instance” functions altogether, there’s no longer a way to update an object in-place. Instead, developers should capture the return value of every function, which will contain a new version of a given object with the updated data returned by the API call.
# Old method
myShipment.Buy(); # myShipment updated behind-the scenes
# New method
Shipment myPurchasedShipment = myClient.Shipment.Buy(myShipment.Id, rate)
# or
myShipment = myClient.Shipment.Buy(myShipment.Id, rate)As noted in the example above, developers can choose whether to assign the return value to a new variable, or re-assign the existing object instance to the return value if they no longer need the old instance.
Moving forward
The Developer Experience team is proud to have finally tackled this design challenge in all seven of our client libraries (and finally been able to close those outstanding GitHub issues!). While we would have preferred to address it without an architectural change, knowing the extra work our end-users will need to take on to adapt to our new design, we feel confident this was the right decision in the long run for supporting these libraries and squashing our thread safety bugs.
All of our client libraries received major version bumps in recent weeks and months as the new design rolled out, and each respective GitHub repository has an Upgrade Guide walking developers through how to migrate their implementations. If you run into any issues, please don’t hesitate to open an issue on our GitHub page or contact our support team at support@easypost.com.
Although we strongly encourage developers to keep their EasyPost library dependencies up-to-date, if you wish to stay on the current version you are using, please pin your dependency to avoid an accidental upgrade.