December 18, 2019

Simplest CQRS Implementation: Improvements and better practice

In my previous post I wrote a very simple implementation of the CQRS pattern and shared it via Gitlab; it was purely an illustration for the purposes of the blog post, and therefore it lacked some of the niceties that you’d expect in a “real” codebase.

Let’s jazz the example up with linting, proper comments/documentation, unit testing, and automated builds. As in the previous example, you may want to keep up by browsing the repository and checking out any of the commits that are referenced.

Linting

The difference that linting can make to a codebase is incredible: it improves readability, can improve documentation (i.e via validating the accuracy of DocBlocks), enforces idiomatic practices, and ultimately improves developer morale!

Go - like most languages - has a dedicated linting tool: golint. Lets take a peek at what is returned if we run this on the codebase in it’s current state:

➜  cqrs-example git:(master) $ golint ./...
audit/audit.go:8:2: exported const AuditActionCreate should have comment (or a comment on this block) or be unexported
audit/audit.go:13:6: exported type AuditEntry should have comment or be unexported
audit/audit.go:13:6: type name will be used as audit.AuditEntry by other packages, and that stutters; consider calling this Entry
audit/audit.go:20:1: exported method AuditEntry.TableName should have comment or be unexported
audit/commands.go:9:6: exported type CreateAuditEntry should have comment or be unexported
audit/commands.go:13:1: exported method CreateAuditEntry.Do should have comment or be unexported
audit/handlers.go:9:1: exported function HTTPHandler should have comment or be unexported
audit/queries.go:7:6: exported type AuditEntries should have comment or be unexported
audit/queries.go:7:6: type name will be used as audit.AuditEntries by other packages, and that stutters; consider calling this Entries
audit/queries.go:11:1: exported method AuditEntries.Get should have comment or be unexported
dispatcher/dispatch.go:10:2: exported var Database should have comment or be unexported
dispatcher/dispatch.go:14:6: exported type DispatchableConfig should have comment or be unexported
dispatcher/dispatch.go:18:6: exported type Command should have comment or be unexported
dispatcher/dispatch.go:22:6: exported type Query should have comment or be unexported
dispatcher/dispatch.go:26:1: exported function Dispatch should have comment or be unexported
message/commands.go:9:6: exported type CreateMessage should have comment or be unexported
message/commands.go:13:1: exported method CreateMessage.Do should have comment or be unexported
message/commands.go:28:6: exported type UpdateMessage should have comment or be unexported
message/commands.go:32:1: exported method UpdateMessage.Do should have comment or be unexported
message/commands.go:43:6: exported type DeleteMessage should have comment or be unexported
message/commands.go:47:1: exported method DeleteMessage.Do should have comment or be unexported
message/handlers.go:10:1: exported function HTTPHandler should have comment or be unexported
message/message.go:7:6: exported type Message should have comment or be unexported
message/message.go:13:1: exported method Message.TableName should have comment or be unexported
message/queries.go:7:6: exported type MessageEntries should have comment or be unexported
message/queries.go:7:6: type name will be used as message.MessageEntries by other packages, and that stutters; consider calling this Entries
message/queries.go:11:1: exported method MessageEntries.Get should have comment or be unexported
➜  cqrs-example git:(master) $

Oh dear - that looks bad, right? Fortunately most of the issues are the same, and they pertain to a lack of comments on exported variables and structs. This is easy to fix, albeit it may be quite tedious. (Lesson: write comments as you go along!)

However, after this commit, we’re greeted by a far more pleasant output:

➜  cqrs-example git:(master) $ golint ./...
➜  cqrs-example git:(master) $

Unit Testing

Although we’re aiming to make this codebase “more realistic”, there’s still not that much to actually unit test: the Command and Query structs only have wrappers around actual gorm functionality. We don’t actually do any processing.

This means that our primary candidates for unit testing are:

  1. HTTP Handlers: do the correct methods generate the correct commands/queries? are invalid methods rejected? are valid payloads returned?
  2. Dispatcher: are invalid Command/Query structs rejected with the correct error? are the correct methods called on valid ones?

One of the most dangerous pitfalls to stumble in to when trying to write unit tests is paying too much attention to test coverage. As a metric test coverage can be useful when applied to core areas with business and domain logic, but the simpler the codebase, the less parts which will actually provide useful opportunities for testing.

We’re already starting at a disadvantage too, and although I’m far from an evangelist of test-driven development, writing tests after writing the implementation does often lead to a degraded quality of tests. This is because you’re only going to capture the behaviour you’ve actually implemented, and that sometimes means missing edge cases, or - more dangerously - capturing any failures as false-positives.

Testing HTTP Handlers

We currently call dispatcher.Dispatch(interface{}) directly from our HTTP handlers, this poses as a bit of a dilemma for testing - as we’re using our concrete interface.

func HTTPHandler(w http.ResponseWriter, r *http.Request) {
    // ...

    query := &ListingQuery{}
    if err := dispatcher.Dispatch(query); err != nil {
        w.WriteHeader(http.StatusInternalServerError)
        return
    }

    // ...
}

To get around this, we’re going to (a) create a new type for our dispatch function, and (b) wrap our HTTP handlers in closures, so we can inject the dispatch function in to them. This is simple and requires minimal changes.

// dispatcher/dispatch.go
type DispatchFunc func(interface{}) error

// audit/handlers.go
func HTTPHandlers(dispatch dispatcher.DispatchFunc) http.HandlerFunc {
    return func(w http.ResponseWriter, r *http.Request) {
        // ...

        query := &ListingQuery{}
        if err := dispatch(query); err != nil {
            w.WriteHeader(http.StatusInternalServerError)
            return
        }

        // ...
    }
}

Our tests can now initialise the handlers with a mock version of the dispatcher.Dispatch function! With this in mind we’re going to use the Go standard library to write tests for each HTTP verb supported by the handler, we want to check (a) the type of Command or Query that’s dispatched, and (b) the HTTP Status Code returned.

These tests can be expressed in an idiomatic tabular format, and given that there’s currently no validation of payloads, they can be written quite concisely. Here’s the example for the audit package:

func TestAuditHandlerAllowedMethods(t *testing.T) {
    handler := HTTPHandlers(func(_ interface{}) error {
        return nil
    })

    testCases := map[string]int{
        http.MethodGet:    http.StatusOK,
        http.MethodPost:   http.StatusNotImplemented,
        http.MethodPut:    http.StatusNotImplemented,
        http.MethodPatch:  http.StatusNotImplemented,
        http.MethodDelete: http.StatusNotImplemented,
    }

    for method, expectedStatusCode := range testCases {
        req, err := http.NewRequest(method, "/audit", nil)
        if err != nil {
            t.Fatal(err)
        }

        rr := httptest.NewRecorder()
        handler.ServeHTTP(rr, req)

        actualStatusCode := rr.Result().StatusCode
        if actualStatusCode != expectedStatusCode {
            t.Errorf("incorrect status code returned by audit endpoint (%s:%d)",
                method, actualStatusCode)
        }
    }
}

Usually I’d recommend using the excellent testify/assert package for making assertions in unit tests.

The message package is more complex, with more test cases, but still follows a similar pattern; you can check it out here.

Testing our dispatcher

Our dispatcher is incredibly simple - in part because it doesn’t dispatch, it just calls a provided message that’s available on the Command or Query. (Remember, there’s no need to complicate things more than they need to be!)

With this in mind we want to ensure:

  1. Given a Command, the Dispatcher will call Do;

    a) and a configuration struct - of type DispatchableConfig - will be provided via the Do call.

  2. Given a Query, the Dispatcher will call Get;

    a) and a configuration struct - of type DispatchableConfig - will be provided via the Do call.

  3. Given a struct that is neither of type Query or Command, the Dispatcher will return ErrorInvalidCommandOrQuery.

These are nice and simple, for example - this is the test for criteria number 3:

type invalidInput struct{}

func Test_DispatcherWillErrorOnInvalidInput(t *testing.T) {
    invalid := &invalidInput{}
    err := Dispatch(invalid)

    if err != ErrorInvalidCommandOrQuery {
        t.Errorf("expected ErrorInvalidCommandOrQuery, got %s", err)
    }
}

For all the tests, be sure to look at the dispatch_test.go file - or specifically commit ba5e9439.

Continuous Integration and Automation

With safeguards in place for both code readability (via linting) and functional correctness (via unit testing), we now need to ensure that we have the infrastructure in place to automatically verify that our code satisfies these tests. For this, we turn to Continuous Integration.

Things begin to get a little more difficult here, if only because it depends upon the specific CI solution you’re using. As I’m using Gitlab to host this example, I’m going to write a very simple .gitlab-ci.yml file that utilises their built-in CI solution.

As we’re looking purely at the codebase, and not deployment, I’m going to ignore the actual deployable build - which would most likely be a Docker image. Instead all we’re going to do is (a) install any dependencies for linting/testing (i.e golint), (b) run the linter, and © run the unit tests.

These three tasks can be achieved in this rather small .gitlab-ci.yml file:

stages:
  - test

test:
  stage: test
  image: golang
  script:
    - go get -u golang.org/x/lint/golint
    - golint ./...
    - go test ./...

You can see a passing build for it here - Job #383978625, and the actual introduction of the file is in commit ba5e9439.

Final Thoughts

This post took the simple example application from the previous one, and added some safety and quality checks to it - making it much more akin to a real life project.

I’ve been able to write these two posts relatively quickly as Christmas is approaching and I’ve opted to take some time out for myself, focusing on some much needed personal time and pet projects. I hope to write one more part (and I have a pretty specific idea in mind!), whereby I implement a feature properly using a tests-first workflow. Alas, we shall see if that actually gets written..!

© Fergus In London 2019

Powered by Hugo & Kiss. Source available on Github.