Engineering Blog Atom Feed

Bardia Keyoumarsi

Clean-Go for Web Services

by Bardia Keyoumarsi

Here at EasyPost we utilize the standard constructs within the Go language to set up our services with Clean Architecture in mind. In this article we will walk you through a Go web service setup and exercise a dependency migration that showcases the advantages of Clean Architecture.

Prerequisites

There are plenty of articles on the world wide web about the Clean Architecture philosophy. We suggest that a brief review of them should help you better understand the concepts used in this walkthrough.

Here is one that I have utilized in the past:

Web Service Setup

We typically break our service setup into four distinct segments:

  1. Repository packages that provide data access interfaces
  2. The entity package that implements the data constructs shared between packages
  3. Service packages that provide application/business logic
  4. External interface packages (HTTP controllers or CLI commands)

For our example walkthrough, we are going to set up a service that performs some specific business logic based on data that it receives from its repo packages. The service directory would look like the following:

service directory structure

The api and cmd packages house the external interfaces into our service. The only logic they contain is the process of receiving the external request (HTTP request, CLI command invocation) and setting up the expectations to use the business logic interfaces defined in the service packages.

As previously mentioned, the entity package holds the entity contracts between the repo, service, api, and cmd packages. This allows the application to scale its external interfaces independently of the intra-service dependencies.

And last but not least, we have the repo and services packages that respectively hold the data access and business logic interfaces.

Now for our example scenario, let's set up a repo package that handles event creation. Our initial datastore is a MySQL cluster.

    
    package event

    type Repo interface {
        // CreateEvent stores the provided event details 
        // and returns a populated Event entity
        CreateEvent(params CreateEventParams) (*entity.Event, error)
    }
            
    type mySQLRepo struct {}

    func NewRepo() Repo {
        return &mySQLRepo{}
    }

    func (r *mySQLRepo) CreateEvent(params CreateEventParams) (*entity.Event, error) {
        // code to insert event record into MySQL backed database
    }
    

As for the service package, let's set up a capture service that takes some inputs from the api package, performs some business logic related to capturing events, and finally calls the repo/event package interface to store the event data in the data store.

    
    package capture
    
    import "repo/event"

    var eventRepo = event.NewRepo()

    type Service interface {
        // CaptureEvent ensures the user account is configured 
        // for event capture before storing the event data
        CaptureEvent(params CaptureEventParams) (*entity.Event, error)
    }

    type captureV1 struct{}

    func NewService() Service {
        return captureV1{}
    }

    func (s *captureV1) CaptureEvent(params CaptureEventParams) (*entity.Event, error) {
        // verify account configuration and other business logic
        // finally, when ready to store the event data:
        return eventRepo.CreateEvent(createParams)
    }
    

Backend Datastore Migration

Now this is where we outline the effectiveness of the Clean Architecture setup and using interface data objects for our Go service.

Let's assume that we have reached a boiling point with our MySQL datastore and after exhaustive research we have determined that we need to migrate our datastore to a Cassandra backend.

Our migration plan could look something like:

  1. Set up a Cassandra backend store
  2. Have the service dual-write to both Cassandra and MySQL backends
  3. Verify data consistency across both data stores
  4. Flip the switch and only use the Cassandra backend

Let's set up repo/event for dual writing to complete step 2 of our migration plan.

    
    package event

    type Repo interface {
        // CreateEvent stores the provided event details
        // and returns a populated Event entity
        CreateEvent(params CreateEventParams) (*entity.Event, error)
    }

    type mySQLRepo struct {}
    type cassandraRepo struct {}
    type dualWriterRepo struct {
        sqlRepo mySQLRepo
        xandraRepo cassandraRepo
    }

    func NewRepo() Repo {
        return &dualWriterRepo{
            sqlRepo: mySQLRepo{}
            xandraRepo: mySQLRepo{}
        }
    }

    func (r *mySQLRepo) CreateEvent(params CreateEventParams) (*entity.Event, error) {
        // code to insert event record into MySQL backed database
    }

    func (r *cassandraRepo) CreateEvent(params CreateEventParams) (*entity.Event, error) {
        // code to insert event record into Cassandra backed database
    }

    func (r *dualWriterRepo) CreateEvent(params CreateEventParams) (*entity.Event, error) {
        r.xandraRepo.CreateEvent(params)
        return r.sqlRepo.CreateEvent(params)
    }
    

Now we have our code setup so that our events are stored in both MySQL and Cassandra data stores. This change is fully isolated to the repo/event package. Neither the service nor the api/cmd packages are affected by this change.

After gaining confidence in our Cassandra integration, it is time to flip the switch fully over to Cassandra insertions only. This change is going to be as simple as removing the now deprecated dualWriterRepo and mySQLRepo implementations. The NewRepo() function will now just return a pointer to cassandraRepo{}.

    
    package event

    type Repo interface {
        // CreateEvent stores the provided event details 
        // and returns a populated Event entity
        CreateEvent(params CreateEventParams) (*entity.Event, error)
    }

    type cassandraRepo struct {}

    func NewRepo() Repo {
        Return &cassandraRepo{}
    }

    func (r *cassandraRepo) CreateEvent(params CreateEventParams) (*entity.Event, error) {
        // code to insert event record into Cassandra backed database
    }
    

Other Advantages

Beside clean migrations of integrations across the service components, there are other positive gains from this approach.

Take unit testing for example. With your service components isolated in dedicated packages and use of interface{} methods, we can easily setup mock interfaces for the package dependencies. This way, the unit tests for the individual package are focused on itself and we have the ability to enforce various edge cases through the use of the mock dependencies.

How about introducing new API releases? Want to change your API input parameters? Gopher it! The above approach should allow you to make isolated changes within the api package. The different versions of your API then transform the input parameters into the required entity construct used between the service components internally.

Conclusion

There are many ways you can set up your service structure. Ultimately, the project alongside the short-term and long-term needs should inform the approach. At a glance, the Clean Architecture approach may seem to be a bit more work than other approaches but the extra work pays dividends over the long-term, especially when scaling the application is one of the criterias.