sf softwareist

An introduction to YAIA

2019-07-20T00:00:00+00:00

Photo by DDP on Unsplash

With so many iOS architectures out there these days, I was sure that one would speak to me. I’ve tried Viper, MVVM, ReSwift and others. They each had their merits but ultimately their shortcomings prevented me from using them. So what did I do?? I Invent one more architecture! And what is the worst name I could come up with for it? Glad you asked! I call it “Yet Another iOS Archtecture” or YAIA (pronounced ‘Yay-yeah!’).

Goals of YAIA

An architecture should have goals, otherwise its just a collection of arbitrary rules which is no fun. Below I’ll describe some of the values that motivated YAIA.

Writing unit tests should be painless. This is the main goal of YAIA and everything else falls out of that. I’d like to write test without mocks, fakes or injecting anything. I’d like to avoid testing async code and I certainly don’t want to do any IO such as reading files or hitting a database in my tests. I’d like my test to run as quickly as possible.
The architecture should be flexible and light weight. It doesn’t require implementers to create classes X, Y and Z and protocols for each of those classes (I’m looking at you, Viper!). I also don’t want tests to implement stubs or mocks to facilitate testing.
Changes should be easy to make. This is related to points one and two. I don’t want to discourage users from making improvements because it will require changes in many places or might break something.
Simple. No unnecessary abstractions or classes. Abstractions such as protocols or base classes can make your code flexible but they also make it harder to follow. Every class or protocol has to earn its place. If it doesn’t, it’s out.

To achieve these goals, I cherry-picked features from a few different architectures I had used in the past. From ReSwift, an implementation of the redux pattern, I borrowed its emphasis on value types and keeping data and logic in one place. In YAIA, the models that represent the business and app logic are all value types (as opposed to reference types). The inputs and outputs to this values-only system are also values. This makes results easy to verify by simply making them Equatable. A nice side effect of a values only system is that it’s impossible for it to contain any async code paths (a struct or other value type can’t be captured by reference in an escaping closure). This keeps tests simple and running fast.

While there are many things to dislike about Viper, one of its tenants is that services should be hidden behind interfaces. This is one place where an abstraction can be nice. Types from services shouldn’t creep into the business logic. What this means for iOS is that Core Data’s NSManagedObjects, wont be referenced in the values-only core. Instead, we copy data out of Core Data and into plain old Swift structs. These value type are then passed into our system. This allows us to test our system without instantiating Core Data objects. It also means that I can switch persistence technology without too much pain since the Core Data dependencies are contained; our business logic layer will be unaffected when we change the type of database we’re using.

A brief overview

I’ve described this architecture in previous posts but I’ll go over the highlights here. For a given screen you’d like to build, a login screen perhaps, the class that ties the different elements together is called a Scene (naming things is hard). In this simple example of a login screen I’ve called it LoginScene. It behaves similarly to the ViewModel in MVVM. In YAIA, it has a weak reference to the view controller via a protocol. This is one of the few places that I use a protocol in YAIA. The view controller has a strong reference to the Scene so that the Scene’s lifecycle is tied to the view controller’s.

The Scene object contains a State object which is some sort of value type; generally a struct or enum. I will discuss the State object further but for now understand that the bulk of the logic resides here. The Scene object may also have network or database services as dependencies. The Scene’s job is to move data between the State object and these dependencies (the UI is also a dependency).

It is the Scene’s responsibility to take input from the UI, a button press for instance, and pass it to the State object for processing. At this point, the State object might update its internal state and possibly return a value type indicating the IO to be performed. The Scene would then tell the appropriate interface to perform the request (eg network request). The result of this request is then fed back into the State for processing. If the State object determines a view transition is required, it will return an enum representing that transition which is then passed to a flow coordinator which handles the transition.

To update the UI, the State will return a ViewModel instance which is passed to the ui by the Scene. In YAIA, a ViewModel is a simple data transfer object (a value type of course) that contains all the info the view needs to update itself with.

But how does all this help me?

How does YAIA achieve the goals listed above? Since the main goal is to make software testable, I had to figure out what type of code is easily tested. I discovered that if you separate the logic and state from the IO, the state/logic portion becomes much more testable. The easiest thing in the world to test is a function that takes values in and returns a value. I wouldn’t hesitate to write a test for func add(a: Int, b: Int)->Int. It takes values as inputs and returns a value. It doesn’t have any internal state, doesn’t read from a database and doesn’t have any dependencies that make the tests complicated. A test based on this method would be easy to read and debug.

The opposite end of the spectrum is a class which handles all of its IO internally and has its logic and IO coupled together. This class may be dependent on other complex classes which must be recreated in a test environment or mocked. When testing this code, the setup ends up being many times longer than the actual test and verification portion of the test. This means the tests are brittle and hard to read. Additionally, since they take so much effort to configure, developers will be less inclined to write tests in the first place. By making the State object a value which doesn’t do any IO, we side step most of the difficulties above.

To achieve flexibility, I recommend using your best judgment regarding when to use the pieces above. If you’re building a screen with some text and a single button, put all the logic in the view controller and move on. This is just one approach to separating logic from IO but there are other ways to achieve the same goal. When it comes to architecture, one size does not fit all.

In the end, the specific names of the components and the exact responsibilities of a given class don’t matter. What matters it being able to write software that is easily verified by unit test. This is achieved by isolating logic in value types and away from the async code and IO. By doing that, you are one step closer to making tests work for you.

Neovim and Swift, a Match Made in Heaven

2018-12-05T00:00:00+00:00

Photo by Justin Greene on Unsplash

Every once in a while two technologies will converge, yielding a sum that is greater than their parts. If you are lucky enough, you will witness just such an event once or maybe twice in your life. Swift LSP combined with Neovim is one of these cataclysmic events. Hyperbole aside, I’m really excited about the prospect of effectively writing Swift outside of XCode. Once the project was announced, I couldn’t wait to dive in and integrate it with my favorite editor.

What is Swift LSP?

Recently the Swift team announced that they would be releasing an LSP server (Language Server Protocol) for Swift. An LSP server, developed for a particular language, allows any editor which implements an LSP client to attain many IDE like features for editing that language, for free. These features include code completion, formatting, refactoring, go to definition, find references, contextual documentation look up and much more.

Before LSP, each IDE or editor had to implement all the above features for each language they supported. As a result, most editors would have a patchwork of functionality implemented for the few languages they supported. Polyglot developers would have to learn a new IDE for each language they used or do without advanced editing features.

While the Swift team has made amazing progress on the LSP server, it is still an immature project. Currently the only supported LSP features are code completion, go to definition, documentation lookup and find all references. Even though this is only a small subset of what LSP has to offer, it is already a big boost to developer productivity.

How it works

Before I get into how to integrate the server with Neovim, I need to talk a bit about how an LSP server works. An LSP server needs to know how all the source files in a project relate to each other. Therefore, the server must know something about how the project is built. It needs to know which files are included in the project, where to find dependencies and which flags are passed to the compiler. The Swift LSP uses the Swift Package Manager to understand how the project is built. As such, if you want to use Swift LSP, your project needs be a Swift package. I wont go into detail on creating a package but you will need a Package.swift at the root of your project.

You will also need to install the Swift tool chain describe on the Swift LSP website. You will then need to define SOURCEKIT_TOOLCHAIN_PATH in your .zshrc. It will look something like export SOURCEKIT_TOOLCHAIN_PATH=<path to toolchain>

Additionally, we will need an LSP Client. In our case, we will use LanguageClient-neovim (LCN form here on out), a Neovim plugin. Not only will this give you IDE like features for Swift but also for any other language implementing an LSP server.

I’ve created a demo repository that handles installing all the required Neovim plugins, downloads and build the LSP server and implements a small Swift package to experiment with. It can be found here. It uses the Dein plugin manager for Neovim to install plugins. It also uses Deoplete for code completion. LCN will fallback to use omniFunc to do completion so Deoplete isn’t strictly required. The project uses its own vimrc and puts all plugins in the cache folder so your Neovim installation should not be affected.

How to use the demo repo

To use the repository, there are 3 steps:

Clone my repo, cd into it and ./init.sh. This will fetch the sourcekit-lsp submodule, build it and build the demo Swift package found at MySwiftPackage. It will also update neovim with the Dein, Deoplete and LCN plugins.
Download and install the Swift toolchain recommended on the sourcekit-lsp website(12-4-2018 at the time of this writing). Make sure SOURCEKIT_TOOLCHAIN_PATH in tryit.sh points to the newly installed xctoochain file.
Run tryit.sh which will set some environment variables and launch Neovim into the main.swift file. At this point you are ready to start experimenting!
In Neovim, run :UpdateRemotePlugins

Things to try

If you move the cursor over a symbol eg. MyStruct and type gd in normal mode. You will be taken to the MyStruct definition in the Greeter.swift file. gd is mapped to :call LanguageClient#textDocument_definition()<CR>
Open a line under line 4 and type my_array.. You should see a list of all methods and properties defined by Array. You should be able to Tab through all the options.
Move the cursor over first in my_array.first! and type :call LanguageClient#textDocument_hover()<CR>. This should open up a preview window with the documentation for Array.first property.
Type :call LanguageClient_contextMenu()<CR>. This will bring up a context menu allowing you to select from the actions above. Most options are not implemented yet but gives you an idea of what will be possible in the future.
The Swift LSP server docs say that find all references is supported but I wasn’t able to make it work. If you figure it out, drop me a line.

As you can see, the functionality is currently somewhat limited but the Swift team is moving fast. Even as I wrote this new commits were being pushed. Neovim is a pleasure to use and with the advent of the Swift LSP server I could see my self moving to it for all my Swift development needs.

Caveats

There are a few catches that I ran into. First of all, you need to tell LCN how to find the root of the Swift project. In the vimrc file I told LCN to look for a file called Package.swift which indicates the root of a Swift project. By default LCN looks for a .git folder which, in our case, is one level up from our SPM root. Swift LSP also doesn’t index in the background, it will only index a project’s symbols at build time. This means that the LSP server wont be aware of any new symbols added to a project until it is recompiled. The Swift LSP project is moving fast so make sure you always have the latest versions of everything.

Imitation Game, Implementing Dribbble Concepts

2018-10-09T00:00:00+00:00

Photo by Anton Darius | @theSollers on Unsplash

Lately I’ve been learning UI design basics and spending time on Dribbble, a site where designer types show off their latest creations. My first thought when I see an amazing animation is always ‘Whoa, that looks so cool!’ and my second thought is always ‘Whoa, I have no idea how to implement that!’. Studying these animations has inspired me to close the gap between what designers are creating in After Effects and what I’m able to do with UIKit.

In order to hone my skills and challenge myself, I decided to take a design from Dribbble and implement it in Swift. I chose something that wasn’t too complicated and had a variety of animations. The one I chose was the VNPAY concept design by Ramotion. Ramotion does amazing work and they’re SF locals!

This is their final product. It was probably mocked up in After Effects and rendered to a gif.

This app concept is a simple 2 screen interaction. It contains a login view which transitions to a dashboard screen. It utilizes a Google Material Design aesthetic, in which interactive controls appear to float above the rest of the UI. I particularly like the animated blue stripe on the login screen so this is what I implemented first.

I admit that I picked this interaction partially because I wanted to learn how to use Lottie and I thought the stripe animation would be a good first experience. I was seriously impressed. Briefly, Lottie enables exporting vector based animations from After Effects into iOS or Android. This means you have all the power of AE animations in your iOS projects. Learning After Effects and setting up animation exporting consumed almost half the time I spent on this project but it was worth it. My blue stripe doesn’t do justice to Lottie’s power. Head on over to their site to see some of Lottie’s amazing abilities (but not yet!).

I sped up the boring bits (like my slow typing), but the animations are running at normal speed.

To the left you can see my implementation. For the rounded controls and buttons, initially I had integrated material-components-ios. This is Google’s implementation of Material Design for UIKit and is a rather large library. I could imagine using it if I was going all in on Material Design. For this project, however, it seemed a little heavy handed as I only needed a few shadows and rounded corners. In the end, I ended up dropping it and rolling my own components.

Next, to implement the actual animations, I tried Stellar. I had high hopes for this library — It was well documented, had a nice interface and seemed quite powerful. Unfortunately, I struggled to integrate it with my project, the maintainer had stopped supporting it a few Swifts ago. Additionally, it was difficult to make simple animations work properly and, most importantly, it didn’t take advantage of the more modern UIViewPropertyAnimator API which I was interested in learning.

Eventually I ended up dropping Stellar and creating my own animation library. Né Disco. Disco is a thin wrapper around UIViewPropertyAnimator. It simply adds a fluent interface to property animator. It’s pretty bare bones at the moment but I intend to flesh it out when I have some time.

Since we’re on the topic of libraries I created for this project, I’d also like to introduce Choreo. Choreo doesn’t do any animation on its own. It simply choreographs the animations between several views. Users of the library can specify the amount of time the entire animation should take, the start and duration time of each view’s animation. Choreo will start each view’s animation at the proper time. Like Disco, I intend to spend more time polishing this up in the future.

Some challenges I encountered during this project were getting the animation timing functions to match the gif. I broke their gif down into individual frames to calculate the length of a each animation but had to guess at their timing curves. I also never figured out exactly what typeface they are using.

This ended up being a really fun project. I learned a lot about CoreAnimation and UIViewPropertyAnimator and got two new libraries out of it! In the future I intend to do another one of these little experiments, but for now I’m switching gears and am attempting build my daughter some sort of electronic toy. Wish me luck!

WWDC Refactored

2018-06-20T00:00:00+00:00

Recently, I’ve been discussing ways to architect iOS applications to make them easier to test. Yesterday, I stumbled upon this talk from WWDC ‘17. In this video, the presenter espouses a lot of the same ideas I’ve been advocating here. It’s a great video and I highly recommend it.

There, the presenter refactors a method that might be seen in any application. The original method would be hard or impossible to unit tests due to its dependency on the UIApplication singleton. They refactor it using several techniques, including mocking a dependency, in order to make the logic testable. The result is a much more testable unit of code. Here is the original method:

@IBAction func openTapped(_ sender: Any) {
  let mode: String
  switch segmentedControl.selectedSegmentIndex {
    case 0: mode = "view"
    case 1: mode = "edit"
    default: fatalError("Impossible case")
  }
  let url = URL(string: "myappscheme://open?id=\(document.identifier)&mode=\(mode)")!
  if UIApplication.shared.canOpenURL(url) {
    UIApplication.shared.open(url, options: [:], completionHandler: nil)
  } else {
    handleURLError()
  }
}

And here is the refactored version:

protocol URLOpening {
  func canOpenURL(_ url: URL) -> Bool
  func open(_ url: URL, options: [String: Any], completionHandler: ((Bool) -> Void)?)
}
extension UIApplication: URLOpening {
  // Nothing needed here!
}
class DocumentOpener {
  let urlOpener: URLOpening
  init(urlOpener: URLOpening = UIApplication.shared) {
    self.urlOpener = urlOpener
  }
  func open(_ document: Document, mode: OpenMode) {
    let modeString = mode.rawValue
    let url = URL(string: "myappscheme://open?id=\(document.identifier)&mode=\(modeString)")!
    if urlOpener.canOpenURL(url) {
      urlOpener.open(url, options: [:], completionHandler: nil)
    } else {
      handleURLError()
    }
  }
}

And this is how you would test it:

class MockURLOpener: URLOpening {
  var canOpen = false
  var openedURL: URL?
  func canOpenURL(_ url: URL) -> Bool {
    return canOpen
  }
  func open(_ url: URL,
      options: [String: Any],
      completionHandler: ((Bool) -> Void)?) {
      openedURL = url 
  }
}
func testDocumentOpenerWhenItCanOpen() {
  let urlOpener = MockURLOpener()
  urlOpener.canOpen = true
  let documentOpener = DocumentOpener(urlOpener: urlOpener)
  documentOpener.open(Document(identifier: "TheID"), mode: .edit)
  XCTAssertEqual(urlOpener.openedURL, URL(string: "myappscheme://open?id=TheID&mode=edit"))
}

This is a huge improvement over the original method. It allows us to test all the logic contained in DocumentOpener. It decouples the application singleton from the logic under test. My only objection is that the test is a little convoluted. As someone unfamiliar with the code, I need to examine the mock object to see how it works and I need to open up the DocumentOpener to understand how it interacts with the MockURLOpener. Additionally, the URLOpening protocol makes the production code harder to reason about. Was the protocol added to facilitate testing or did the writer truly intend for consumers of DocumentOpener to implement multiple URLOpening classes? What if we refactored this using some tips I’ve outlined in previous articles?

class DocumentOpener {
  enum Result {
    case openURL(URL), invalidURLError
  }
  let isURLValid: (URL) -> Bool
  init(isURLValid: (URL) -> Bool) {
    self.isURLValid = isURLValid
  }
  func open(_ document: Document, mode: OpenMode) -> DocumentOpener.Result {
    let modeString = mode.rawValue
    let url = URL(string: "myappscheme://open?id=\(document.identifier)&mode=\(modeString)")!
    if isURLValid(url) {
      return .openURL(url)
    } else {
      return .invalidURLError
    }
  }
}

Arguably this is a little nicer; we’ve removed a few lines of code and a type declaration. The real value comes when attempting to test this unit.

func validURL(_ url: URL) -> Bool {
  return true
}
func testDocumentOpenerWhenItCanOpen() {
  let documentOpener = DocumentOpener(isURLValid: validURL)
  let result = documentOpener.open(Document(identifier: "TheID"), mode: .edit)
  XCTAssertEqual(result, .openURL(URL(string: "myappscheme://open?id=TheID&mode=edit")))
}

As you can see, we are injecting a single method instead of a conforming object. Additionally, we are returning a value representing the action we’d like to perform. The advantage is that our test is reduced from 18 lines to 8, there is almost no arrange code, and the return value is easily verified.

This isn’t always the best approach. If DocumentOpener had a lot of interaction with the system via URLOpening, we’d end up injecting a lot of methods into DocumentOpener. At this point it might make sense to inject a URLOpening object instead.

In conclusion, by avoiding mocks and focusing on value types, it is possible to write tests that are shorter and easier to understand down the road.

Pt. 3 Swift Testability By Example

2018-05-31T00:00:00+00:00

In my last article, I discuss the easiest path to testable Swift. In that article I list qualities that make tests valuable. Additionally, I show that business and application logic should be decoupled from volatile or asynchronous dependencies. Now I’d like to focus on the “State” object that houses all of this logic and illustrate some design decisions that will make it easier to test.
An experienced tester knows that certain features are trivial to test while others must be mangled and distorted before they yield to testing. Tests that are easy to write are ones that require little or no arrangement code, don’t require new classes to enable testing (ie. mocks or stubs) and produce easily verifiable output. The easiest thing in the world to test is a pure function:

let sum = addInts(3, 4)
XCTestAssertEqual(sum, 7)

We dream of tests that look like this. A pure function is one who’s return value is solely determined by its input parameters. Its internals don’t reference any global mutable data which can affect the return value. For a given input, it will always return the same output. Its output is another value, not a write to a database or network request. The output is a value that can be easily verified using the == operator.
The next easiest thing to test is a mutable object that has internal state but doesn’t reference any volatile dependencies. To test these objects we must first construct the object, then mutate it until it is in the state we’d like to verify and finally check that certain conditions are true. An example of this would be:

var calculator = Calculator(startingValue:0)
calculator.press(key: .three)
calculator.press(key: .plus)
calculator.press(key: .four)
calculator.press(key: .equals)
let finalValue = calculator.currentValue
XCTestAssertEqual(finalValue, 7)

Again, this class (or struct) doesn’t make any network calls and doesn’t reference any global data; calling these methods, in this order, with these parameters will always result in the same final value. It’s not quite as simple as the previous example but it is still obvious what this test does and why.
We only want to write tests that resemble these two forms. Unfortunately, most unit tests I’ve seen in Swift follow a different pattern.

class MockAPIClient: APIClient {
    var error: Error?
    func make(request: APIRequest, onComplete complete:(Error?) -> ()) {
        complete(error)
    }
    //Add all other methods required by APIClient,
    //possibly stubbing return values to make
    //MockAPIClient behave like the real APIClient
}
class MockLoginDelegate: LoginViewModelDelegate {
    var showErrorBannerCalled: Bool = false
    func showErrorBanner(withMessage: String) {
        showErrorBannerCalled = true
    }
    //Stub all other LoginViewModelDelegate methods
}
let mockDelegate = MockLoginDelegate()
let apiClient = MockAPIClient()
apiClient.error = //some error
let viewModel = LoginViewModel(apiClient: apiClient)
viewModel.delegate = mockDelegate

viewModel.userTappedLogin(username: "steve_jobs", password: "i<3apple")
XCTestAssertTrue(mockDelegate.showErrorBannerCalled)

I avoid this style because it violates a number of rules outlined in my previous article.

First, it is long and it would be a lot longer if I had implemented every method in LoginViewDelegate and APIClient. Long tests force the reader to figure out what code is required by the test versus what is there just to keep the compiler happy. They are also brittle; every line must be maintained and can break if an interface changes. Besides, who wants to type that much.
In order for LoginViewModel to function properly, MockAPIClient must return the proper values from all its public methods. For instance, if hasNetworkConnection: Bool is declared in APIClient, we need to grok the implementation details of LoginViewModel in order to know that this method must return true for the tests to behave properly. Any time a test is coupled to implementation details like this, it’s sure to cause trouble.
In order to inject the MockAPIClient, we are forced to add the APIClient protocol which wouldn’t otherwise be there. This protocol needs to be updated whenever RealAPIClient‘s public interface changes. This forces all of the tests to be updated as well. Protocols add a level of abstraction that can make code harder to reason about; it’s unclear to the reader that APIClient will always be a RealAPIClient at runtime.

Converting the above to a more simplified form gives us the following:

let loginState = LoginState()
let error = /*some error*/
let commands: [LoginState.Command] = loginState.handleLogin(response: nil, error: error)
XCTestAssertEqual(commands, [.showErrorBanner("Failed to log in")])

Four lines of code, no mocks or stubs and no protocols added. Value types are passed in and value types are returned. The production code might look like the following:

class LoginViewModel {
    var state = LoginViewState()
    let apiClient = APIClient()
    func loginTapped() {
        let commands = state.loginTapped()
        self.handle(commands: commands)
    }

    func handle(commands: [LoginViewState.Command]) {
        for command in commands {
            switch command {
            case .sendLoginRequest(let request):
                apiClient.send(request: request) { response, error in
                    let commands = self.state.handleLogin(response, error)
                    self.handle(commands: commands)
                }
            case .showErrorBanner(let message):
                self.ui.showErrorBanner(message)
            case .showLandingScreen:
                self.delegate.transitionToLandingPage()
            }
        }
    }    
}

disclaimer: I have no idea if any of this compiles

I’ve left out some details for brevity but you can see how the logic of what to do is embedded in the state object while how to do it is found elsewhere. Of course, this is a toy example but you can imagine the state object will become more complex as requirements are added (eg. validation logic, error cases). This will be contained and kept separate from the IO represented by the Command objects. You can find more examples of these patterns here.

In this article I’ve outlined some specific examples of code that is easily testable. I’ve also given some pointers on how to restructure code to be tested painlessly. Thanks for reading!

Edit: Here I’ve been carrying on about separating logic and IO, using value types and avoiding mutation like it was some great discovery but turns out apple outlined a lot of these ideas at WWDC 2017. 🤦‍♀️ If you haven’t caught that talk yet, give it a watch. It’s full of good information.

Pt. 2 Testable Swift

2018-05-08T00:00:00+00:00

In my last article I touched on a few ideas to make Swift testable. Here I’d like to demonstrate these ideas by implementing a hypothetical feature. The feature is written using TDD and exhibits these virtuous patterns. Before I go on, I’d like to describe the attributes which make a test good.

What to Strive For

Tests should be easy to read. All setup code is located in close proximity to the test condition. It is clear what the setup code does and why it is required. The test is concise and only includes code relevant to the test. Readers can easily grok which product requirement the test demonstrates. Testing enables others to understand the code as much as it enables the writer to create it.
Tests should be easy to write. Only a few lines of code are required to arrange the test. The test executes synchronously and no mocks or stubs are required to implement them. If there is friction in writing tests, developers will skimp on them.
Tests should not make production code more complex. Adding abstractions or making private APIs public to facilitate testing muddies the intention of the code. Tests enable developers to write simpler production code.
Tests should not be brittle. As long as the API under test doesn’t change, wild changes to the implementation can be made and test failures will only be seen when a feature is broken. The confidence to make dramatic changes is the true power of unit testing.
Tests should execute quickly. While practicing TDD, the tests are run dozens of times a day. If you must wait for the simulator to spin up each time, momentum will be lost.

Sounds nice doesn’t it? Most people who dip their toes into the TDD end of the pool have the opposite experience. Testers create several mocked classes and write tens of lines of setup code for each test. The suite runs painfully slowly making TDD impossible. Changes to the system break or invalidate dozens of tests, making refactoring harder instead of easier.

An Example

My approach to this article is to assign myself a hypothetical feature and implement it using TDD. I followed a few heuristics which make it much simpler to write testable code.

So what is this secret testing sauce I keep alluding to? The idea is to keep the logic and state isolated from the IO or any other volatile dependencies. By ‘logic and state’ I mean business rules and domain models, I also mean the state required by a view controller, or the queuing and prioritization logic required by a network controller. In this instance ‘IO’ means disk and network access, UI interactions, anything to do with GCD as well as DispatchQueues and even timers. As long as all IO have been removed from the code under test, writing the tests will be simple.

The feature I’ve decided to build is a BLE pairing screen. The user is shown all broadcasting BLE devices in their vicinity. They are able to tap a broadcasting device and give it a nickname which adds it to the local Core Data store. In this scenario I have three IO devices; the CBCentralManager, the CoreData repo and the view controller displaying the list of BLE devices. I also added cell animations when a users adds, deletes or renames a device. The complex logic behind this feature is where TDD really shines.

I want to stress that this is not another “MV* architecture will fix your problems” article. The example here is just one implementation illustrating these principles. There are many ways to isolate logic from volatile dependencies. These guidelines apply to all software project, not just view based ones.

Bringing It All Together

In the diagram below, I’ve indicated how the IO devices integrate with the UI logic:

As you can see, the red box is the system under test (SUT). It contains all of the classes that we are testing for this screen. At first glance, it might appear that we’ve failed to test a great deal of the system. On closer inspection, we can see that all the interesting logic and data structures are contained solely in the red box. The green box is only responsible for shuttling data back and forth between the IO devices and the state object. It has a cyclomatic complexity of one. The three IO devices in the orange box may contain their own logic and state (queues and retry logic etc) but would be covered under a different set of tests.

An Aside on ViewControllers

The view controller on the bottom right is especially interesting. We are so used to the view controller being the real meat and potatoes of any screen, its hard to imagine it not containing anything worth testing. In this example when something in the model changes, a new TableViewModel is pushed to the view controller along with a RowAnimations object (both structs) describing the animations. All the view controller does is copy values out of that model and into the table view cells and run the animations. Again, there is no logic in the VC, it has no knowledge of the underlying data it represents, it simply updates itself with the model which is pushed to it. The result is that the VC is easy to reuse.

There is no logic in the TableViewModel, it is simply a data transfer object. The values in these data transfer objects are what we will be verifying in our tests.

In Conclusion

Using this approach won’t uncover every conceivable bug in your project. There will be issues integrating the repository, the BLE interface and the UI. We will cover the interaction of those components using integration tests. If each component is rigorously unit tested, we will be more confident when we go to integrate them.

In my next post I’ll illustrate some patterns and heuristics I use to keep Swift testable. See you next time!

Software Intimacy

2018-02-07T00:00:00+00:00

Enlightenment is intimacy with all things. —Ehei Dogen (1200-1253)

What is intimacy in software? It’s a funny concept which I’ve wanted to write about for some time. Like interpersonal intimacy, it’s hard to define. So, instead I will illustrate it with a few examples.

The first time I lacked intimacy with a codebase was on a project in which I had been asked to do some backend work. The server engineers set up my PHP environment and, to my chagrin, they weren’t using any type of debugger. Their process was to make a change locally, push it to develop and then test the feature using curl. If it didn’t work as expected, they would add some print statements, ssh into the server and tail the logs. Ouch! I felt like I was observing the code through a straw.

With a setup like this, it is hard to be sure what the server is doing at any given moment. I could only observer the values for which I had had the foresight to print beforehand. Many debugging insights occur by chance, through observing values in the debugger. All the properties look normal, but “oh wait, this property over here looks off” which leads us in a new direction. These chance observations are less likely to happen if we are required to add print statements apriori. Limited debugging combined with the tedious, ‘edit -> upload -> test -> tail the logs’ cycle means that we are much less likely to find the root cause of an issue, and more likely to add band aid fixes.

Once we were able to debug a running process over a socket, and could watch the program as it ran, our feeling of intimacy with the project grew. We could see the details of the running program more clearly now. By watching a program run in a debugger we learned how the program worked more innately. This kind of intimacy is especially important when learning a new codebase.

The next change to my process that increased my intimacy with a codebase was when I added unit testing to my toolkit. Using the debugger allows us to view a process as it runs; similarly unit testing allow us to see how our code behaves under different conditions. I like to think of my test suite as a little laboratory where I can test my subjects under tightly controlled conditions. Not only do I know how my code behaves under normal circumstances, now I know how it behaves under error cases and other obscure scenarios. I also know that all the assumptions I’ve made about the code still hold true after a change. Knowing that things still behave properly after a big refactor is a huge confidence booster.

The most recent change I’ve made to my process, and one I’m still experimenting with, is doing UI development in Playgrounds. The ability to observe the effect of a code change immediately in the interface is very powerful. With a properly factored UI, it is possible to inject all sorts of data and see the effect in real time.

My normal flow when building UIs was to do layout in interface builder, compile, run, navigate to the screen I was building and observe my changes. If I wanted to test an edge case, I would hardcode some data and feed it to the view. Once I had verified that the UI behaved as expected, I would remove the mock data. Adding hardcoded values is tedious and not easily reproducible.

Now, I load my view in a playground, along with different sets of mock data, and can instantly see my changes as they’re made. This technique is a great way to observe how the UI will behave under different conditions. It also speeds up the UI development process since I don’t have to navigate to the screen I’m building. Playgrounds are still in their infancy but in the future, they will be a useful development tool in additions to being a great learning device.

It isn’t possible to observe what a computer is doing directly, so developers have created powerful tools to see how things behave at runtime. By using these tools we can become more intimate with our software. This allows us to easily debug issues as they arise and refactor aggressively to prevent them from appearing in the first place.

A lack of intimacy motivates developers to make smaller local changes in order to fix issues instead of making larger refactors which will make the codebase more coherent. Knowing intimately how code behaves gives developers the confidence to make larger changes that improve the codebase, making it more maintainable.

The take home message is to embrace tools that help you understand what your software is doing and integrate them into your daily process..

Pt. 1 Effective Unit Testing in Swift

2017-10-04T00:00:00+00:00

I’m coming clean. I’m embarrassed to admit it, but I’ve been a professional software developer for twelve years and have hardly written any tests. I’ve embraced the unit testing cult a few times, but every time I did, I spent more time hacking my code to make it testable than writing new features. My velocity went into the toilet, my once beautiful code became riddled with concessions to make it testable, tests would break after small changes, and worst of all, they never seemed to surface any genuine defects. I would eventually give up with the excuse that “iOS code isn’t testable” or “unit tests are for people who write bad code”. Such hubris!

The cycle would repeat itself several times over the course of my career. Eventually the proponents of unit testing got louder and louder, even smart people in the iOS community. Clearly I was missing something. By this point I had been consulting for a few years, and had worked with several companies that enforced test coverage. They all suffered from the same problems I did. Everyone was recommending tests but few were doing it effectively.

At some point I stumbled upon the Boundaries and Functional Core, Imperative Shell talks by Gary Bernhardt. Finally, I found someone else who understood my pain and, more importantly, had a way forward. I consumed all material I could find relating to the subject and found more people espousing the same ideas. The venerable Uncle Bob discusses similar concepts in his Clean Architecture talks and book.

At a high level, these speakers introduced several ideas that were new to me: Mocks should be avoided, logic and I/O should be separate, source code dependencies (#imports) should point from implementation details like the network or database toward the core business and application logic, and that the core should be more functional than OOP. Many of these ideas seemed counter to common practice.

For the last decade my architecture hadn’t fundamentally change. Classes perform some logic and then write data to a database or network. They were difficult to test because data in the DB would have to be verified or network connections mocked in order to verify what was posted to the server. These tests involved a lot of arrangement code before the actual test, it wasn’t obvious what they were doing and they would often break after small changes.

The problem was that I was testing too many layers of my app. I hadn’t separated my logic from my controllers, network/database or UI. To verify the logic I’d have to fake interactions from the user and verify requests made to the backend. I would need mock users, mock authentications and mock network requests. I was trying to test my code as one big lump. The unit tests had become quasi integration tests.

If you fundamentally separate the code into two different categories these problems disappear. One side is made of structs and other value types. It contains all the business and application logic. It does no I/O, and has little or no external or volatile dependencies; which is to say it makes no contact with the outside world. It is functional in nature and doesn’t make any asynchronous calls. This half of the source code represent the app’s ‘state’; the part that changes over the lifetime of the app. It is what you write unit tests against, nothing else.

The other side is comprised of controller, coordinator and manager classes. It handles all of the network, database and interface I/O (yes, the screen is an I/O device). The dependency arrow points from these classes to the logic code above. The goal is to have no logic here whatsoever. It’s purpose is to marshal data and events back and forth to the logic level. Ideally it has a cyclomatic complexity of 1 (meaning it contains no conditional statements) and no state at all. These classes are not unit tested. They are simple to the point that they can be verified by looking at them and do not change often.

The logic layer can be tested at whatever granularity you wish. It is comprised of simple value types so there is no need to mock anything. No unnecessary protocols or interfaces are needed to facilitate testing. Values are much easier to verify than database writes or JSON blobs. These tests run faster since they aren’t writing to a database and don’t depend on any app scaffolding. They can be run without a simulator, meaning they complete in milliseconds instead of seconds.

The implication is that nothing under test will have dependencies on UIKit (or any other *Kit). There wont be any tests against views or view controllers, but that’s ok, the action isn’t located there anyway. As an example, instead of instantiating a view controller, passing mock data to it and verifying a label’s text property, tests would be written against a view state struct that would have a text property on it. Tests would verify this property instead. It is true that it’s impossible to achieve 100% test coverage with this approach but there are diminishing returns in trying to cover every single line.

Mock objects are a useful tool but must be used conservatively and with care because they have some considerable drawbacks. Mocks make tests brittle since they must be updated whenever the interface they are mocking changes. They are complicated to arrange before each test and many times they require their dependencies to be mocked as well. They also trigger more false positives since they don’t always behave the way the real object might. Another drawback mocks have in strongly typed languages like Swift is that they require a protocol where you wouldn’t otherwise need one. Unnecessary abstractions make code harder to reason about and is an example of test induced design damage.

By keeping the dependency arrows pointing from the controllers toward the business logic, we ensure that our logic is testable and dependency free. This is achieved through dependency injection; the controller implements a protocol defined by the logic layer. To test the system at a macro level, simply mock and inject the I/O controller. This is one place where mocks are useful.

That’s enough for today. Thanks for taking the time to listen to my thoughts. The take away is to separate the logic from other aspects of the source code, if you find yourself writing too many nested mocks you may have taken a wrong turn and make sure the controller/manager/coordinator classes are dependent on the logic objects and not the other way around.

Obscure Programming Laws And What They Can Do For You

2017-08-22T00:00:00+00:00

In my last post I explained some of the common anti-patterns that developers fall into. In this article I’d like to explore some architectural principles that help make code more robust and extensible. Like all ‘rules’ in software development, these are really only guidelines, so if something feels awkward or forced, it probably is.

Law of Demeter

The first principle I’d like to discuss is the Law of Demeter. Demeter is the Greek god of software development (that’s not true). Demeter’s law or “The principle of least knowledge” provides guidelines on to which object another object may access. It states that an object ‘A’ implementing a method ‘m’ may invoke methods on: 1) itself (‘A’), 2) any objects passed as a parameter to ‘m’, or 3) any objects instantiated by ‘A’, either as a property or as a local variable. Instances of ‘A’ may not interact with any other objects; specifically ‘A’ shouldn’t invoke methods on objects returned from calls to other objects.

As an example, something like self.myObjectB.getC().doSomething() should be avoided. Instead you might do something like self.myObjectB.doSomethingToC() (I hope you’re a little more creative with your method naming).

At first this seems highly restrictive but the advantage is that you’re limiting the dependencies of your object; ‘A’ no longer needs to know anything about ‘C’. ‘C’ could be easily refactored or removed from the codebase and ‘A’ would not need any modification. The “surface area” of an object is comprised of all its methods plus all of the methods of any objects it exposes publicly. By making ‘C’ private and modifying it through ‘B’s public methods, B’s surface area has been reduced. This makes it easier to reason how changes to ‘C’ will affect the rest of the codebase. Again, this is only a guideline and wont always be appropriate.

DRY

Ok, this one isn’t very obscure but it is often overlooked. It stands for Don’t Repeat Yourself. It means that redundant functionality should be factored out. Code gets duplicated when a feature is implemented in one part of the code, and then similar functionality is needed elsewhere. Instead of making the initial feature generic so that is will work in multiple places, the original implementation is copy/pasted to its new location, and potentially modified slightly for its new purpose. The problem is that a bug in the original code must be fixed in two places. If additional functionality is added to one implementation, the other doesn’t reap the benefits. Code can be made more versatile in a number of ways. Reusable code can be achieved through generic/template functions if your language supports them, by extracting the generic functionality into its own methods and passing in the unique functionality via function pointers or closures, or by putting the generic code into a base class with unique functionality added to derived classes.

KISS

KISS stands for Keep It Simple, Stupid. While it sounds a little harsh, it is actually good advice. There is a tendency for developers to think their job is done once the code functions as designed. In reality working code is just the first step. From there the code needs to be boiled down and simplified. Are you using two data structures when you could accomplish the same thing with one? Are some properties of a class redundant? Has some operation been repeated in several places (i.e. is it DRY)? Should some logic be wrapped in its own class? Could a block of code be shortened and its intension be made more clear? By taking the time to simplify and refactor new code once it is working, the code will be easier to reason about and modify in the future.

The Open/Closed Principle

This principle states that classes and functions should be “Open for extension but closed for modification” meaning that it should be possible to extend the functionality of an existing class without having to open it up and modify it directly. An example of a class that violates this principle would be a ShapeRenderer class which draws different shapes to the screen and has the functionality for drawing each shape (drawSquare(), drawCircle() etc) added as methods to the ShapeRenderer. Any time a new type of shape is needed, the ShapeRendered class must be opened up and modified. What if rendering to a file is needed? The ShapeRenderer class would need to be modified for this as well. The problem is that the developer needs to understand the implementation details of the ShapeRenderer class completely in order to modify it. A better ShapeRenderer would be one that accepts different shape objects to be rendered. Each shape object would know how to render itself and ShapeRenderer would not have to know the specifics about each shape. To add a new triangle shape, ShapeRenderer wouldn’t have to be modified at all, the triangle drawing code would be implemented in Triangle, a subclass of Shape.

Now lets say that we want to be able to render our shapes to a file instead of to the screen. The ShapeRenderer class could be modified to take a RenderTarget object. The RenderTarget subclass would be responsible for actually writing the renderings to the screen or to a file. This would make it simpler to add another render destination, such as a printer. A RenderTarget subclass would contain all the printer specific code. No need to modify ShapeRenderer.

Single Responsibility Principle

This principle states that a class should “have one reason to change”. I always had trouble with this definition. A class could have any number of reasons to change. The description seems too vague to be useful. To adhere to this principle, you kind of have to follow your instincts. Classes always have responsibilities, they are in charge of certain functionality. Ideally each class is only in charge of one specific task or a small collection of related tasks. If a class is in charge of multiple unrelated things it should be split up and they should communicate through some sort of interface. The problem with having 2 orthogonal pieces of functionality in the same class is that their implementations become intertwined. It becomes difficult to reason about one without understanding how the other works. Their implementations become tightly coupled and it becomes difficult to modify one without breaking the other. A good heuristic is that if a subset of methods and properties in a class all deal with one aspect of its functionality, they might be a good candidate for pulling out into their own class.

Conclusion

That is the short list of principles I’ve used to help guide my work. There are many others but these are the ones which resonate with me and I hope you find them useful as well. As a word of caution, I find that whenever I learn a new paradigm or discover a new axiom, I tend to over apply it, so like everything in life, finding balance is key.

iOS Code Smells I’ve Smelt — Common Pitfalls of a Junior iOS Developer

2015-05-28T00:00:00+00:00

Being a freelance iOS developer, I get the opportunity to work on a variety of projects, written by developers with varying degrees of experience. This has given me the opportunity to catalog the most common iOS anti patterns or ‘Code smells’ I’ve smelt. I will enumerate the most common smells and some possible solutions to guide you on your path to mobile app nirvana.

MVC

The most common error that junior developers make is constructing the ‘Massive View Controller’ and its cousin, the ‘Massive App Delegate’. People commonly place bits of logic in these two classes because they don’t know where else it should go. If this pattern isn’t curtailed early on, view controllers can become 400 (4000?) line behemoths. This is a sign of an unhealthy code base because the code isn’t easily reused, leading to code duplication and unnecessary dependencies. The other, more insidious problem is that, inevitably, your view controller will implement multiple systems in the same class, their logic will become hopelessly entangled and difficult to reason about. Attempting to modify one system can easily introduce bugs in another. Any given class should do only one thing and have a clean interface exposing its functionality.

To keep view controllers slim, remember they should only be responsible for transferring changes from the model layer to the views, or passing user input back to the model layer. View controllers should be the most boring classes in your project. Similarly the app delegate should simply relay system messages to the data layer. When it comes down to it, the reason these two classes become so bloated is that they are the only classes that Xcode generates automatically for you. The solution is to simply pull out a piece of paper and draw some boxes representing your classes. Then write down a list of all the public methods that each class exposes. After that, draw some arrows showing their dependencies to each other. Do this before writing a single line of code and your project will be more modular and flexible.

APIs define explicit relationships between objects

The second stinkiest smell I’ve smelt has many different manifestations but all share a common root. I’m going to call this one the ‘Lack of Well Defined APIs’ smell. People think an API is something you slap on top of a library right before you ship it. On the contrary, any object you create should have a carefully considered interface before work begins on the implementation. If your project has even two classes, each should have a well defined list of public methods describing the functionality of the class. For bonus points it should even have all of its dependencies exposed in the constructor (see Dependency Injection). The approach new developers take is to dive right into the implementation and not consider how classes relate to one another.

There are many different forms the ‘ill defined API’ code smell can take. One example is a custom table view cell which exposes all of its subviews to the world. Users of this custom cell are expected to access the labels and buttons of this cell directly. Instead, all access to the subviews should be through an API. This makes the code easier to read, explains exactly what the cell does, and allows the developer to change the implementation of the cell without changing any other class. In the example below, I don’t need to change any code outside of the cell when I later decide to use use the ‘attributedText’ property instead of the ‘text’ property in the cell’s highScoreLabel.

//This is a leaky cell API
cell.highScoreLabel.text = [NSString stringWithFormat:@“%@ %d”, user.name, user.highScore];

//This method abstracts away the implementation of the cell, leading code that is easier to modify
[cell setName:user.name highScore:user.highScore];

Another common example of a poor API is when embedding a button in a cell, often developers will set the target/action pair on the button in the data source. My approach is to expose a block in the cell’s interface which gets executed when the button is tapped. This allows me to capture the data object that the cell represents in cellForRowAtIndexPath:.

//This is a leaky abstraction
[cell.favoriteButton addTarget:self forAction:@selector(toggleMarkFavorite:) forControlEven:UIControlEventTouchUpInside]

//A better alternative
cell.favoriteButtonTapped = ^(BOOL isFavorite) {
if (isFavorite) {
    [cellObject markAsFavorite];
} else {
    [cellObject unmarkAsFavorite];
}

In the same vain, modifying the UI by walking the view hierarchy, is a design anti pattern. Code that either iterates over subviews calling “isKindOf:“ searching for a particular view, or searches up the hierarchy calling superview.superview, is a code smell. Making assumptions about the structure of a view hierarchy is extremely brittle and will easily break with simple changes to the view structure. I once worked on a project where a previous developer had referenced table view cells by calling superview.superview from an embedded subview. Once iOS 7 rolled out, apple changed the internal structure of the cell and ALL THE CODE BROKE. The solution is to add direct references (properties) to any views which need to be modified at run time through Interface Builder.

Another symptom of poorly considered object relationships is excessive use of NSNotifications. Notifications are, in and of themselves, not a bad thing and do have many legitimate uses. However using them to facilitate communication between two objects instead of a direct dependency is a code smell. At first glance, Notifications make writing code easier; changes to one object are easily propagated to other objects without defining an explicit relationship between them. The down side to this approach is that the relationship between these two objects isn’t obvious by looking at their public APIs. It’s better to make a direct relationship between two objects, either through a ‘has a’ relationship, the delegate pattern or a more specific observer pattern (blocks also facilitate interaction between objects but see my next point).

Too much of a good thing

When blocks were first introduced, I thought they were the best thing since the ‘for’ loop. I quickly discovered that, while blocks make asynchronous code easier to read, they became difficult to follow and debug as they become more deeply nested. The beauty of blocks is that you can capture the state of your program at any point and do operations on it later. The problem is that it can be difficult to determine the order that operations occur if too much of your code is executed in blocks. Don’t get me wrong, blocks are a good thing and are absolutely appropriate for many circumstances. However, whenever creating a block nested inside another block, ask yourself if there is a way to combine them. Alternatives to blocks are: the delegate pattern, an observer pattern, NSOperations or a Promises library.

Polymorphic views

Xib files make interfaces easy to assemble but can make reusing components difficult. This gives rise to the ‘overloaded view’ code smell. This happens when you define a custom view, such as a table view cell, in IB. Then elsewhere in the UI you see a similar view, not quite the same but similar, so you decide to reuse the same xib file, possibly passing in a ‘type’ enum to the cell to differentiate between the two. The view shows and hides various subviews depending on it’s ‘type’. These view ‘types’ seem to grow in number and are only superficially related to one another. Code paths which are unrelated accumulate in one place and become hopelessly entangled. The result is one class representing many views in your app which might share some minor commonalities but, in reality, have very little to do with one another. You know there is a problem when the view is littered with switch statements; adding and hiding views depending on some state. This is a problem because code written for one ‘type’ can inadvertently get executed for a completely separate type. It also makes the code harder to reason about and debug.

A better approach is to factor out the common components into a separate .xib file. The xib then gets added as a subview where needed.

Just make it an object already

A smell I see quite often, especially when working with RESTful APIs, is the ‘Dictionaries as Objects’ smell. JSON objects, received from the server, are parsed intro dictionaries and used directly throughout the app. The problem is that dictionaries are stringly typed; as in you have to use strings to access the values. This makes the code verbose and can lead to bugs if keys are misspelled. It is also impossible to attach functionality to a dictionary to process the data like you would with methods on classes. Additionally the kinds of values you can store in a dictionary are restricted (no primitive types). It is preferable to define classes for all JSON objects and convert the dictionary responses to objects as soon as you receive them. The advantage is that you can access properties using getters instead of keys, you can use primitive types as values and you can you can add functionality to the data if need be.

Ya ain’t gonna need it

Poor grammar but good advice. A lot of code I come across is littered with commented out code or worse, functions and classes that aren’t used anywhere but were left in the code base, collecting digital dust. The habit of developers to save code because ‘they might need it later’ is fairly pervasive. The problem with stale, unused code is that it still needs to be mentally parsed and understood when reading the code and still needs to be maintained when refactors occur. It is difficult to remove later because a method might appear to be used, but in reality might actually be called from an unused branch of code. If some functionality has been removed, just delete all the code associated with it. It will exist forever in version control in case it is needed in the future. Furthermore, code is cheap to write and expensive to maintain. No matter how brilliant you think your snippet of code is, just delete it, it could probably be recreated later if necessary.

Caching considered harmful

The final code smell I would like to discuss is the dangers of caching. Most developers have a good idea of what caching is; they could probably give an example involving local storage of downloaded images. It’s important to recognize the subtler forms of caching and how they can be detrimental. Some examples of subtle caching are: 1) Performing a calculation at startup and saving the result, 2) creating an object once and using it for the lifetime of the app, 3) fetching objects from Core Data and keeping them in memory, 4) pulling objects from the server and storing them in Core Data. The main problem with these sorts of caching is that now you have two places where the data is stored and they can become inconsistent. The complexity in determining when a cache is stale and needs to be refreshed can introduce bugs to a code base. Before deciding to cache or ‘memoize’ a value, be certain that you need the performance boost, otherwise you’re optimizing prematurely. This anti pattern crops up a lot with Core Data, or when constructing singletons (unless you truly have some state that needs to be maintained for the entire life of your app, it’s better to create new *Manager objects for every view controller that needs access to them). Also, consider keeping data pulled from the server in memory and refresh it when the app launches.

Conclusion

I’ve outlined the seven stinkiest code smells found in iOS projects. Taking the time up front to structure your code properly will always pay dividends down the road.Refactor old code, design clean APIs, create new classes when needed, avoid premature optimization and generally keep your project clean and lean, and your codebase will remain flexible and less error prone. New features will be added easily and old ones will be modified with little friction. The source of and solution to bugs found in your code will be apparent and easy to fix. If you can avoid these code smells, you’ll be miles ahead of your peers. Happy coding!