Project Idea: Smarten up the smart thermostat.

I’ve been messing with Homekit recently.

More specifically I’ve been looking at using Homebridge, a JS package that provides a way to connect your own devices to other Homekit stuff.

Last year I wrote a thing that used a Particle Photon to get the temperature in my living room, and then some Ruby code that ran on a Raspberry Pi to turn on and off my window unit. Essentially a thermostat since the one built in to the unit doesn’t really work.

There are a couple of examples of hooking a Photon into Homebridge, so that whatever data they’re collection shows up inside your Home app.

I like this idea a lot, since it opens the door to using a combo of store-bought sensors and custom made ones. Also, I had to reverse engineer the API (which would make a great blog post) that the Wifi Smart plug my A/C unit is plugged into uses. If I could get things working inside of Homekit then I wouldn’t have to depend on their private & proprietary API not changing.

I would also like to have my Ruby code check more info that just the inside/outside temperature. I’d like it to also look at the UV index, as that can have quite an impact on the temperature indoors. If I got the temperature shoved into Homebridge, and the Smart Plug in there as well, it’d be a lot more manageable code-wise to add another dimension to my K-Nearest Neighbors that I’m using to decide if I want A/C or not.

I was reading a machine learning book when I wrote the code and wanted an excuse to implement something. So the “Should the A/C be on” question is answered by using K-Nearest Neighbors based on past times I’ve specified that the A/C should be on.

So I want to add the UV index to that, and probably a look-ahead to the next hour on the forecast. Currently when the day is warming up the A/C will kick on and off until the middle part of the day at which point the A/C kicks on and the room is too hot for it to cool down. I’d like it to go ahead and notice that it’s going to be too warm later on.

So that’s:

move sensor readings to Homekit via Homebridge
add UV index to my sensor readings
add UV index to my decision matrix
add forecast look-ahead to the decision tree

I’ll need to look in to weather the look-ahead would work well as just adding it as a dimension or if I should just perform the calculation again with the forecast temperature. I kind of like the latter.

Evented Programming in Ruby

So most people when they think of Evented programming in Ruby, their mind immediately jumps to EventMachine, or HTTP requests, or maybe if they’re really a nerd SAX processing XML documents.

EventMachine really stole the show when it came to the scene and to my knowledge just became the de facto for anything event based. When Node came around the scene seemed to change to “if you want to do use event loops use JavaScript”.

But those are far from the only places that events are useful in Ruby.

One of my favorite lessons taken from Elixir was the solid emphasis on sending messages. In Elixir, it’s a requirement that you send messages to talk between processes. It’s the only way for one process to talk to another. By requiring processes to communicate through messages, it repeatedly hammers the idea of clearly defined interfaces between processes into the forefront of the developers mind.

Elixir does all this because Erlang’s lightweight processes and functional nature make processes the way to maintain state and organize your application.

The fact is though that all of the best lessons from organizing code using processes and messages are also really awesome when applied to Object-Oriented code in Ruby. Thinking of “calling functions” as “sending messages” is an often touted idea among the best OO devs, so this isn’t anything new. There’s a reason for it. Building code by designing interfaces brings the same benefits to any OO paradigm that Elixir takes advantage of because of the rules forced onto it by Erlang.

In Objective-C, Cocoa makes heavy use of defined interfaces and delegate objects. Throughout building a Cocoa app you will create several objects and set a delegate property that will be used to process callbacks and customize behavior or the object.

Relying on this requires having a well defined set of public interfaces that delegates can implement and an equal set of notifications that are sent by objects.

This paradigm shifts the focus from “calling” functions that manipulate state to sending and receiving messages that react to events in the system.

Shifting this focus places a greater emphasis on building well documented, flexible APIs. Inside my own code, if I can make a system that has a defined interface to maintain whatever state in necessary, I’ve won.

Living this world allows you to leverage events that are already happening in your system. Currently Rails makes use of this in a handful of ways. For instance, the different gems that Rails provides use ActiveSupport.on_load to take care of initial setup tasks like adding configuration options that each one provides. When you require "active_record", it adds an on_load event listener that gets triggered when some of the base classes get loaded. This makes it possible to both defer configuration and take advantage of eager-loading.

Rails also uses as separate implementation of this in its fantastic instrumentation setup. Rails uses its instrumenters internally to generate most of the awesome log statements you get in development. All of those awesome “Rendered User#index” with timestamps for how long was spend in controllers/views/sql all use this sort of internal pub/sub. In ActiveSupport::Notification there are examples of how to hook into the default events as well as how to publish your own. It’s worth a read.

Taking advantage of this in my own code means that I’ve had to put enough thought into it that I’ve picked out the events that I need and provided some appropriate interfaces for interrogating the data that I have.

Since I can’t really explain these things without example code, here’s a project where I took advantage of this.

I was writing some code that needed to check the temperature at an interval as well as respond to bluetooth button presses and POSTs over HTTP. I also wanted to provide access to the state of the system so that I could check and make sure it’s running correctly. I also needed to keep track of the state of an external system (a window unit A/C).

There’s a gem called EventBus by Kevin Rutherford that allows you to register objects to receive events and provides an interface to publish events to them.

To do this I ended up with three events:

tick: for triggering the periodic temperature check.
toggle: for triggering a change of state.
state_changed: for tracking the change of state. These pass a to argument.

The objects I ended up with were:

StateManager, which subscribed to state_change events to maintain a record of state (on/off).
TemperatureManager, which responds to tick. If a tick is received, it checks the temperature and will publish a toggle message the temperature is too high and the A/C needs to be turned on.
A Sinatra app that publishes a page. If a POST /toggle is received it publishes a toggle message.
an AirConditioner object that responds to toggle messages and turns on and off the A/C. If it’s successful in changing it, it publishes a state_changed message that StateManager will get.

In the end, we have a handful of events and objects all with clear responsibilities and interfaces. Each object is then free to handle those events as they come, or not depending on how that object decides.

Having those events defined also made it easy to add functionality. I went out of town and wanted to have it send me push notifications when my A/C was turned on. I already had a defined place to add that. I simply added another object, a StateChangeNotification that subscribed and responded to a state_changed message. Without changing any of the code that actually made the app function I was able to add a whole new feature.

The most obvious critique is that this could also be done with just OO, and that’s 100% valid. But by setting these constraints I force myself to thoroughly explore & design the objects that I end up creating. And in a language that can really encourage just throwing state around places.

So, this is a somewhat off the wall example. Most of the time I’m not writing code to control my air conditioner.

Most growing Rails applications make use of Rails’ ActiveJob to defer processing of taxing tasks. ActiveJob, and the older Resque or Sidekiq or whatever, are basically a defined way of sending messages without being explicit about sending messages. To an large extent, it’s a bit of an unhelpful abstraction, because for the large part the way I’ve seen it used forces a paradigm that could be useful internally in a process to only be used for inter-process jobs.

I don’t really have an end for this. I could keep rambling on about how amazing message passing is, but this is already entirely too long and needs to stop.

Using Object Models over Service Objects

This week at work I ended up having a conversation that I’ve had before about when to use service objects vs. when to use PORO model classes.

I’ve had this conversation before, a few times, and vaguely recalled that last time I was on the other side of it. So I reached out to my friend Scott hoping he could set me straight and he did. I’m going to go ahead and write it up so that hopefully next time I’m in this situation I can just refer back to here.

So, what the fuck am I talking about?

The general consensus around the office was that Service Objects were a fad that flew around the Ruby community sometime around 2014. At the time, I loved them.

Essentially, they provided a place to house “glue code” that you were going to use multiple times, usually stuff that was fairly business logic-y, or stuff that was complex and didn’t quite belong in a model or just in a controller action.

With a definition that vague, how could anything go wrong, right?

So I ended up using them a lot. I used them for things like importers. I would have a beautiful PersonImporter class that would handle things like creating a person with a given set of params. I saw the benefit because this application was creating people both in the controller and in a handful of other places like rake tasks that imported records from other sources. At this time this project also had an “evented model” where different services could talk to each other by publishing events, and some of those events might cause a Person to get created, and so it was great to have a single place that handled translating params, creating a person, validating it, creating related people records (which might involve fetching additional information), etc.

So I liked them. I thought they were a dream.

Essentially, the paradigm it had me adopting was something vaguely bigger and less defined than MVC. I had controllers, which were essentially one adapter to access my service objects (rake tasks and event handlers being two others). My models were strictly related to pulling info from the DB, validating individual record values, and defining relationships between themselves. My views were Collection+JSON serialized using the Conglomerate gem.

A little while into living in this dream world, I went to RailsConf and watched Sandi Metz’s talk about “Nothing is Something” , and like so many others I was wholly inspired to write better, more focused, more object-oriented code. If you haven’t watched that talk, seriously quit reading this and go watch that talk. You won’t even need to come back and finish this blog post because you’ll already know.

I couldn’t figure it out on my own, so I got Scott to sit down and watch the video of that talk. Here is, as far as I can remember, what we came up with.

Essentially, we were using Service Objects to hide procedural code inside our Object Oriented design. Mostly to avoid coming up with the correct nouns. Fucking naming things, right?

I didn’t know how to name an object that imported things outside of verbing it, so I just verbed it and threw it in app/services. Which, like, totally fine. It’s a cop-out, but, seriously fuck naming things.

The problem is that it encourages you to write less object-oriented, more procedural style code. I had a lot of code that looked like this:

https://gist.github.com/jakewilkins/92816efbae675cc3a739583a5703cef0

Which is at least organized. It’s not great, but it’s pretty easy to see how I got here. It’s stuffing the complexity further and further down, hopefully creating a top level that’s straightforward and easy to follow.

The thing is, it would be fairly trivial to take this and make it more traditionally OO.

If we just name it correctly, this same thing can happen and be nicely wrapped in much more familiar OO mindset.

Essentially, my service objects were badly formed wrappers around an object that represented some sort of ExternalModel that I didn’t have named in my app.

To name these models better, let’s have a for instance that I’m important people from IMDB using my PersonImporter.

I could instead have a Imdb::Person, living inside of app/models/imdb/person.rb. In IMDB, people have multiple movies, and I would want to suck those down to. So I could have a Imdb::Movie model stored similarly. When a Imdb::Person needs a movie, it creates and instance of a Imdb::Movie, or vice versa.

Once we have our objects setup, sending the familiar #save message would handle translating those external models into their equivalent internal counterparts.

The benefits here seem kind of small. We definitely haven’t solved all the worlds problems.

But I think there is absolutely a benefit here. We’ve avoided introducing a poorly defined abstraction that we have to deal with for the lifetime of our app. Having that model named correctly clearly defines what it represents. It should be clear to anyone looking that a Imdb::Person represents a person, who has something to do with IMDB. I go back and forth in my head whether #save should be #import. If I figure it out I’ll try to come back to add it here so I don’t forget again.

I think for me, service objects were a necessary stepping stone to get from spaghetti everywhere to something more OO. They did a fine job of centrally locating logic that would otherwise have been spread around my code, some in controllers, some in models, and all leaking bugs.

But ultimately I hope next time I remember a little quicker that naming things correctly is always a good idea and in the end leads to cleaner, clearer abstraction layers.

Growing Elixir Applications

Elixir is an awesome new language with some amazing capabilities. It brings with it the power of an amazing framework in OTP, which has been built and used over the better part of two decades.

On top of this, new frameworks have already been built and are maturing nicely. Pheonix, the web framework, is already an amazing front runner that makes it easy to build an incredibly performant website.

Ecto adopts the Repo pattern for database access and already has great support for a growing number of databases. Postgres has first class support for some of its newest and favorite features. It provides easy wrappers for relations of many types as well as support for using Postgres extensions like jsonb.

All these things make it easy to start a new project in Elixir and get it up and running quickly and simply. Spinning up a new blog, a new chat app with live updates over websockets, or an online store becomes as easy as it is in languages that have been around for years.

A quick aside for the less familiar. Elixir not only comes with easy & free interoperability for any Erlang based library, it has a robust dependency management system called Hex. Hex is like RubyGems and Bundler all rolled up if you’re coming from the Ruby world. It allows you to specify dependencies in a file and does version resolution to make things work. It sticks with a more strict `locking` strategy, where gem versions are hard locked into a lock file. But I’ve digressed.

Elixir already provides most of what developers of much older languages are used to. I struggle not to call them more “mature”, because in my view Elixir is mature because of its toolchain.

As a developer, that toolchain makes all the difference. For starters:

repeatable builds based on locked dependencies
stable, pluggable, easily accessible REPL for trying things out
generators for new projects with:
projects like Phoenix can provide their own templates for generators of their own
prebuilt deployment workflows (some battle proven for years inside Erlang)

All these things that just waste developer time ordinarily are handled. Every hour I spend time debugging an app that worked just fine last time I worked on it is an hour I’ve wasted. Having a prebuilt toolchain allows me to skip that shit and get with building new features.

So, basically, Elixir is magic and it just makes everything perfect and easy and perfect and it’s a magical world right?

Not quite.

Elixir and the Erlang zero-cost pass-trhough means that we have to play by the majority of Erlangs rules.

Erlang is a functional language, and variables are immutable unless explicitly freed.

So we’re functional, not object oriented. Which means that variables aren’t objects and so we can’t have state that mutates.

Which, as we’ll see, is actually pretty fucking awesome.

What all this means is that it will force you explicitly manage state that you need. See, when I came to Elixir from Ruby what I realized is that I was using a fuckload of hidden state. I was using objects to store state without actually thinking about why I needed that state, or how to access it.

By making all of that handling explicit, and it impossible for someone to alter state in a way that you don’t want, a developer has to design her state.

After working this way for a while, I found that when writing Ruby most of my bugs (and usually the most tricky ones to solve) were found in mismanaging my objects state. Something was changing state where I didn’t expect, or some state was getting duplicated instead of altered in a way that it wasn’t falling through.

Quite simply, that can’t happen in Elixir. Variables are immutable, which by definition means that no one is reaching in and fucking with your shit. State is handled explicitly, so when I update it, I know. I have to do it purposefully. Not only that, I have to write code that allows me to manipulate that state.

All this makes for fewer bugs, fewer hours spent debugging, more time building features.

Oh, and how can I forget. As a compiled language, a majority of my stupid typo mistakes get caught at compile time. Now, I know that a lot of people, myself included struggle hard getting used to a compiler. They spend time “fighting” it and think it’s time wasted.

Here’s the thing though, that code wouldn’t have ran in a non-compiled language. It would have failed at runtime, hopefully getting caught in tests, at which time it would have blown up anyway.

I like to not only think of the compiler as my friend but as a baseline level of tests that I get for free. You still need to write the appropriate tests and make sure that going forward you’re doing all the things you need to do, but I don’t need to waste a bunch of boiler tests making sure that my interfaces are adhered to. I get those for free when I get my code to compile.

I feel the need to include several paragraphs of what I’m not saying in regards to test but I’m going to give y’all the benefit of the doubt that you know that I’m still saying you should write tests when they make sense.

Many, that’s a lot of crazy cool shit that you pretty much just get for free by using an awesome language. Congrats for making a good choice!

But, it is still a new language. It provides all the correct flex points for building incredibly powerful software, maintainable, and efficient software.

However, its newness limits the documentation available as well as the guidance provided for growing this software in a way that takes full advantage of all of these characteristics.

That’s what I aim to change. I hope to put together a book and several parts to a series that document best practices for growing an Elixir project past the usefulness of the generators and currently available blog posts.