Building something besides a chat app with ActionCable

Recently I needed to be able to render a PDF for a user asynchronously and let them know when it was ready. I’ve solved this previously using a hand rolled JS polling patch, which was fine at the time but this project is using the latest & shiniest version of Rails so I thought I’d try to ride the Rails and try out ActionCable.

This was my first trip through ActionCable since sitting through DHHs keynote announcing it. At the time I was learning Phoenix and so I was less than blown away by this announcement of something that Phoenix had out of the box since day one.

Getting started proved to be the trickiest part. The documentation is good in spots however there are several places that left me guessing as to how to apply ActionCable to something that was not a chat application.

Luckily for me, for most of this I was pairing with my friend Stephanie who helped me get my head wrapped around this in the beginning and handled most of the initial setup as well as getting got all of this deployed.

To get started Stephanie found Michael Hartls’ Learn Enough Action Cable to Be Dangerous and it was by far the most helpful resource for us. One thing I struggled getting straight was an overview of the all the pieces involved and how they plugged together, so I’ll attempt to provide that later on.

My basic task was to allow a user to request a PDF which would be generated asynchronously and notify them when it was ready.

Starting with the outer layer first. My Connection code sets up the `current_user` pretty much exactly how the docs and blog posts I could find said to do:


# I wasn't entirely sure where to shove this, so it got stuffed in
# app/channels/application_cable/connection.rb
module ApplicationCable
  class Connection < ActionCable::Connection::Base
    identified_by :current_user

    def connect
      self.current_user = find_verified_user
    end

    private

    def find_verified_user
      if verified_user = env['warden'].user
        verified_user
      else
        reject_unauthorized_connection
      end
    end
  end
end

My understanding is that since the WebSocket’s share cookies with HTTP traffic the authentication is handled by the users normal login flow. As long as you are using a wss:// (the extra “s” standing for “secure” or something), you can trust generally that your user is logged in and use the session. So in my Connection I am simply using Devise’s Warden setup to load the User from the session.

And the JS to get your app to start making that connection, again straight from Hartl’s excellent examples and the docs:


// app/assets/javascripts/cable.js
//= require action_cable
//= require_self
//= require_tree ./cable

(function() {
  this.App || (this.App = {});

  App.cable = ActionCable.createConsumer();
}).call(this);

Since I had WebSockets available, I decided to use ActionCable’s @perform function to call a method on my channel to enqueue an ActiveJob rather than submitting an HTTP request. Inside the job, when the PDF was ready and uploaded to S3,  we would broadcast on my channel a signed URL to download it from. Here’s my Channel code:


# app/channels/pdf_channel.rb
class PdfChannel < ApplicationCable::Channel
  def subscribed
    stream_from "pdf_#{params[:report]}_for_#{current_user.id}"
  end

  def enqueue_report_job(data)
    report = Report.find(data['report'])

    RenderReportToPdfJob.perform_later(current_user, report)
  end
end

The subscribe method tells ActionCable which keys this channel is interested in. More on that later. The enqueue_report_job is what our JavaScript will trigger to start the process moving.

Here is my CoffeeScript to connect to it:


App.reportChannel = App.cable.subscriptions.create {channel: "ReportChannel", report: $('a[data-report-id]').data('report-id') },
  anchorSelector: "a[data-report-id]"

  connected: ->
    @install()

  disconnected: ->
    @uninstall()
 
  # Called when the subscription is rejected by the server.
  rejected: ->
    @uninstall()

  received: (data) ->
    if data.error
      return @showError(data)
    @displayPdfLink(data)


  install: ->
    $(@anchorSelector).on("click", (e) =>
      link = $("a[data-report-id]")
      @perform("enqueue_report_job", "report": $(@anchorSelector).data("report-id"))
    )

  uninstall: ->
    $(@anchorSelector).off("click")

That CoffeeScript right there is my least favorite part. I am positive I’m doing something silly, and my sincerest hope is that by being wrong on the internet some kind soul will tell me just how silly I am.

So to my understanding, this is the general layout of what we have just created:

if I could have written it, I would have, but I made the graphic because I couldn't.

For now, ignore Client 2 and Channels B and C, they’re important later.

Client 1 has setup a connection, authenticated by her session, and subscribed to a channel for the specific report she is viewing. She has also registered a click handler for the “Generate PDF” button that will use ActionCable bi-directional trickery to call the enqueue_report_job method on the Channel object. At this point we have all the moving parts linked together.

The trickiest part of the whole process was figuring out the stream_from line. In many of the examples online, you see that line used to setup a stream for a chat room. In Hartl’s example he extends it one step further, showcasing the fact that you can call stream_from multiple times within a Channel.

This in the end was helpful but as multiple calls is not mentioned in the docs it also added to my confusion. Reading the docs I was trying to suss out which pieces were responsible for what. I’m not unfamiliar with WebSocket’s in general and I was trying to map my understanding of them to how Rails is using them.

Mainly, I was trying to figure out why, in the JavaScript when I setup the subscription I only had to specify the Report ID for the Channel, but in the stream_from line I needed to specify the Report ID and the current User ID in order to scope it correctly.

If you’re familiar with Redis PUB/SUBthen it’s pretty simple. Whatever you pass in to stream_from is passed directly to a Redis SUBSCRIBE command, so anything that gets PUBLISHED to that key will be forwarded down that channel.

So stream_from is used solely by the backend to map different Publishers to the appropriate Channels, which are already user specific based on that Users connection.

In Michael Hartl’s examples, this was used to send messages to room specific channels by using stream_from("room_#{params[:room}"), as well as streaming alerts to individual users by using stream_from "room_channel_user_#{message_user.id}".

In our report generation code, we want to stream completion notices to specific users for specific reports. So in our channel code we stream_from a key that specifies both a Report ID and a User ID. In order to do that, our background job has to have access to the User record to so that it can generate the same key.

I’m not sure why I got so hung up on that, but it was the thing that felt the trickiest to me.

So our job issues:
ActionCable.server.broadcast("report_#{report.id}_#{user.id}", reportUrl: url)

Which our Javascript will receive in its received function. As far as I can see, everything that gets sent down the channel will get passed to the same received function, but from the docs:

A channel encapsulates a logical unit of work

So it would seem you’re encouraged to subscribe to multiple Channels if you end up feeling like you’re overloading the received function.

Anyway, in our specific Javascript received, we take the URL for the uploaded Report and replace the “Generate PDF” link with a simple “Download PDF” link, easy peasy.

That’s how you build something besides a Chat App with ActionCable. The most challenging aspect was divining the responsibilities of all the moving parts. The Connections, Channels, Subscriptions, and stream_from all sort of fuzzed together. Once those become obvious ActionCable becomes a nicely organized and very functional solution to sending page updates to clients.

💝😽🎉

1,325 Words

Working with the new Phoenix 1.3 directory structure – A Love Story

Recently I had an opportunity to build a project with the not-yet-released Phoenix 1.3. This minor version bump includes some optional new features that, for me, greatly improved the ergonomics of developing my project. I have no insider info into the project or the motivations behind these changes, but I can say as someone that has worked with Phoenix in fits and starts since its pre-1.0 days that on the whole I really enjoyed them.

Off the top of my head while its fresh here are my thoughts on it. This won’t be an exhaustive list of changes because I’m lazy and on vacation. So I’m just going to pull the most notable features from memory, which I think has its own sort of value in that these were the most memorable to someone focused primarily on developing the Elixir side of an app.

In Phoenix 1.3 the /web folder has been moved inside /lib/<project>/web to be more in line with a typical Mix application. To anyone used to a Phoenix project this would be the most immediately noticeable change. Along with this change, all of your controllers and views will also be namespaced inside of Web. For example, the standard Project.PageController that comes with the generator becomes Project.Web.PageController, and the Project.PageView becomes Project.Web.PageView

My first impression of this change is that Phoenix is trying to become more in line with traditional Elixir/Erlang OTP app structures, including their Supervision tree structure and I support that 100%.

In the keynote where these changes were announced, they talked about visualizing your Phoenix code as just one way to interact with your underlying application, which could have many other ways. This is already true even within Phoenix. If your application has an API and a UI, then you most likely have multiple avenues of achieving the same result. Bringing this out explicitly is a huge win for developers.

One of the ways that Phoenix 1.3 makes this explicit is also the next big change in the app. Now, when you generate a new resources (whether through gen.html or gen.shema [the gen.model replacement]), you also have to specify a Context. From the docs:

The context is an Elixir module that serves as an API boundary for the given resource. A context often holds many related resources. Therefore, if the context already exists, it will be augmented with functions for the given resource. Note a resource may also be split over distinct contexts (such as Accounts.User and Payments.User).

To me, this is where 1.3 really shines. When I was first wrapping my head around it, I tended to use Domain a lot in my head instead of Context, which was helpful to me. When you generate a new resource, the context is also generated for you, along with an outline of the functions that probably belong there.


defmodule Project.Accounts do
  @moduledoc """
  The boundary for the Accounts system.
  """

  import Ecto.{Query, Changeset}, warn: false
  alias Project.Repo

  alias Project.Accounts.User

  @doc """
  Returns the list of users.

  ## Examples

      iex> list_users()
      [%User{}, ...]

  """
  def list_users do
    Repo.all(User)
  end

  @doc """
  Gets a single user.

  Raises `Ecto.NoResultsError` if the User does not exist.

  ## Examples

      iex> get_user!(123)
      %User{}

      iex> get_user!(456)
      ** (Ecto.NoResultsError)

  """
  def get_user!(id), do: Repo.get!(User, id)

  ...
end

This lends itself perfectly to building an application around multiple access points to your data. It’s also something that I haven’t seen in any other frameworks I’ve worked with. This sort of organization is typically left as an exercise for the user.

Here is how the Contexts ended up influencing my app design.

I was building an app that had multiple “accounts” that it needed to track, so I had an Accounts.User, I had a Github.User, and I had a Slack.User, each responsible for storing its own data. Inside each of those contexts were the functions I needed to work with the resources it contained.

For example, I needed to be able log in and register as an Accounts.User with Guardian, so these functions got added to the context:


  def authenticate(params) do
    find_from_auth(params)
      |> validate_password(params["password"])
  end

  def register(%{"password" => pass, "password_confirmation" => conf} = params) when pass == conf do
    do_registration(params)
  end
  def register(params) do
    {:error, "Passwords do not match"}
  end

In my Slack.User, I needed ways to associate it to an Accounts.User and so I had helper functions over there as well. I had function in Github.User for maintaining the link between my user and their accounts api. I also built a had a Settings context for a user, and the Settings context knew how to load the settings applicable to the whatever model was provied. I wanted Slack.Users  settings to be aware of Slack Channels and teams as well as just the user, and the context provided a good place to house these separate semantics.

For me, context’s a very welcome abstraction. In my previous Phoenix project it was always a bit confusing as to whether something belonged in /web or in /lib.  That project grew to be pretty hefty,  and ended up having a lib/data_store/ folder which was vaguely similar to what Contexts provide. What I was reaching for was a place to hold the code that in an OO framework like Rails would be shoved on to the model. I love the Repo pattern that Phoenix uses but I did not love includeing Ecto.Query everywhere that I needed to lookup a record. Contexts provide a clear place for holding that code in an Elixir way.

Taken together, I think contexts and the move of web into /lib/project is a clear win. It leads to a more well organized project and in the end I think it will save many headaches. I think it is a project structure that provides clear avenues for growth. Having that structure by default, rather than solving for a simpler use case, really sets Phoenix apart.

I’m very excited to keep building stuff with Elixir and Phoenix. From an outsiders perspective the team has really taken on the hard challenges head on and really moved forward with them.

That new project I build is a bridge to work with GitHub Issues inside of Slack, you should check it out.

💘😽🎉

1,072 Words

Building a Slack slash command with Elixir and Phoenix

tldr – Elixir and Phoenix make building a Slack Slash Command a breeze with their composable web app style and pattern matching. Checkout the Chat Toolbox (currently in beta) if you want to see it in action and have a way to manage GitHub Issues from Slack

I recently started a new project over at toolbox.chat that allows you to work with your GitHub Issues within Slack. It was a lot of fun to build and Elixir/Phoenix made it a breeze. Here’s a little of what I learned building it out.

A slash command is a Slack extension that uses one a / to kick it off. The app I built implements /issue . When Slack gets one of these it makes an HTTP request to your app and will display the response to the user. Simple enough. When you configure your app within Slack you can specify the URL to post to, and if your app adds multiple slash commands each one can have it’s own URL.

So the first step was setting up the application within Slack to make those requests to my app. This is where I made heavy use of ngrok to be able to have Slack make requests to Phoenix running on my laptop.

Once you have a request, it’s time to get to work figuring out what to do with it. Because I wanted my app to have a single entrypoint, I had some string parsing to do. This is where Plug and Elixir’s pattern matching came in extremely handy.

I wanted to be able to:

  • /issue 3 should show me issue number 3 on the repo I have selected for this channel
  • /issue 3 comment <comment> should make a comment
  • /issue actioncable -- bug should search for issues with “actioncable” and the label “bug”
  • /issue created should show me issues I’ve created
  • /issue elixir-lang/plug -- Kind:Feature should show me issues on the Plug repo with the label “Kind:Documentation”

So I had a fair few paths I needed to cover in my parsing. I ended up with a handful of Plugs that split it up into steps, each one a little more expensive than the last.

We kick off our plug-chain by two plugs that, if this is a slash command, will lookup a user record for the user. If we don’t know who you are, we bounce out of the pipeline and ask you to login or register.

After that, we have a plug that will put a github_client into private so that it’s available based on the user we just looked up.

Now that we know who you are and how to talk to GitHub, we enter into our main preprocessor Plug.

For better or worse I channeled my Ruby Ruleby days and ended with a SlackCommandPreprocessor plug that looked something like:


def call(%{private: %{phoenix_action: :issue}, params: %{"text" => command}} = conn, opts) do
  command
  |> RepoInputResolver.process
  |> existentialism
  |> determine_permission
  |> SlackCommandResolver.process
  |> instrument
  |> display_help_if_errors(conn)
end

To break that down:

  1. RepoInputResolver will determine what repo we’re dealing with. In here I included applying defaulted repos (based on settings stored in Postgres), as well as “guessing” the repo owner if we had something we thought was repo but we weren’t sure of the owner.
  2. existentialism will take what came out of the RepoInputResolver and check to see if it is an exisiting repo or repo/issue id combo.
  3. determine_permission will check to see if the logged in user has admin permissions on this repo. We use this to only display close/assign/tag buttons in Slack if you can actually do those things.
  4. SlackCommandResolver is what takes the info we have here and relates it to an opcode. At this point we have all the info we can squeeze out of the string itself and need to figure out what we’re trying to do before we can parse it further. I made a handful of opcodes that get used in the controller to determine what actions to take.
  5. instrument will stash some metadata in Sentry and Scout so that I can better figure out what happened when errors crop up.
  6. display_help_if_errors will hopefully hint at something fixable if we got this far and couldn’t figure out what is happening. If we have absolutely no idea what you’re getting at we point you to our docs.

So that’s a lot to go into one Plug. This project is still new and growing and so it will probably get split out into a couple ones. But to ramp up it was handy to have it all in once place.

Probably the most fun bits to write were the RepoInputResolver and SlackCommandResolver which has already gone through a few iterations.

At first, RepoInputResolver was just returning a Map, and that map was being fed to the rest of the controller to figure out what we were doing. The problem is, several of the commands require additional state once you know that you’re doing that specific command (e.g. query for people to assign an issue to) and it was getting all messy as far as knowing what info we were working with.

So I wrote a SlackCommand struct that had an opcode and a meta Map that could be used for storing additional data. The SlackCommandResolver looks at what information RepoInputResolver was able to find and based on the availability of a specific non-defaulted repo or issue id etc., as well as typically the first word after that, we’re able to assign an opcode to either list my issues, filter my issues, create a new issue or close an issue or whatever.

I was actually really surprised by how terse Elixir can make this. While most of my lines end up being way over 80 chars once I get all my guards in place, it’s able to take a fairly complex task and simplify it in to a series of 3-line functions. So far I’m quite happy with the result.

This is probably already too long of an article, but I’m not done yet. I’ll have to write another that deals with how Slack will send you interactive message responses and dealing with parsing those and responding into the same message slot.

But for now I’ll leave you with this. Hopefully you found it interesting, and I hope you’ll try out the beta of toolbox.chat and find it useful.

Happy coding.

1,082 Words

Reverse engineering the TP-Link Smart Plug API with Charles

I went to the LA Ruby User Group a while ago and watched a hilarious and pointless talk about automating Pokemon Go to beat all your friends or something.

Except like is so often awesome and amazing about the way the Ruby community encourages learning, there was a nugget in there and that was the amazing app Charles that the presenter used to figure out what requests Pokemon Go was making.

From the site:

Charles is an HTTP proxy / HTTP monitor / Reverse Proxy that enables a developer to view all of the HTTP and SSL / HTTPS traffic between their machine and the Internet.

So essentially, use Charles to spy on shit.

When you launch Charles it will start capturing all the traffic originating from your computer. Initially, if it’s SSL it is just flagged as encrypted. If it’s SSL with SNI then you’ll be able to see the destination hostname.

To solve the SSL “problem”, Charles provides a root cert that you can download and trust which will allow Charles to decrypt those messages as well.

Which is great. Fantastic. And a little terrifying. Now anything on my computer that is connecting to any* website I get to see what is being said.

Which doesn’t really help me since I have an app on my phone I need to spy on.

Happily, Charles has this solved. When you’re connected to Wifi, iOS has a built in screen in settings where you can input information for a proxy. Easy. To get the SSL decoding, Charles also provides a root cert for iOS to install.

Quick note if you’re following along at home: uninstall these certs when you’re done. Reinstall them later if you need them again. I mean, it’s a small attack vector but sheesh why leave it open. It’s devastating if exploited.

So, all that configured and connected, I pop open the app and am extremely disappointed when literally nothing shows up when I toggle the smart plug from my phone.

Zilch. Zero.

As best I could tell at the time (and have since confirmed), if the app and the plug are on the same Wifi then the app talks directly to the plug, and Charles wasn’t spying on local network traffic from the phone.

Which, again, probably a good thing for the world, but nicht zehr gut for my project.

Not one to leave well enough alone, I kept digging.

Since this is getting almost ridiculous as the project itself, I’ll summarize.

I tethered my laptop to my phone, and then set my phone to use Charles, running on my laptop, as a proxy. Which is a giant mess that I can’t even believe worked but it did.

Having done all that, I could now see the traffic running between the control app and the TP-Link servers to toggle the switch. While I was at it I was also able to spy the login sequence so that in the case that the authentication token expired I would be able to duplicate the login flow in my code and grab a new token, which did in fact end up necessary.

In the end, the actual HTTP flow was incredibly simple. All of the toggle requests ended up having a small & concise structure, as did the login flow. A small JSON payload with an “action” key, usually all sent to one endpoint.

The code is sweaty and gross and available here if you want to take a look at it. I make no apologies for how gross it is. It is cooling my face right now as I type. Fight me.

Anyway, it was a fun project. This was the first time I have reverse-engineered (if you can call it that) something that was past “pop open the dev tools and copy as curl”. It is always fun to de-mistify something like this. I feel like I’ve hopped a tiny mental barrier now that I can see requests from apps on my phone and opened the door to a realm of possibilities previously unexplored, by me at least.

😘💕

678 Words

Project Idea: Smarten up the smart thermostat.

I’ve been messing with Homekit recently.

More specifically I’ve been looking at using Homebridge, a JS package that provides a way to connect your own devices to other Homekit stuff.

Last year I wrote a thing that used a Particle Photon to get the temperature in my living room, and then some Ruby code that ran on a Raspberry Pi to turn on and off my window unit. Essentially a thermostat since the one built in to the unit doesn’t really work.

There are a couple of examples of hooking a Photon into Homebridge, so that whatever data they’re collection shows up inside your Home app.

I like this idea a lot, since it opens the door to using a combo of store-bought sensors and custom made ones. Also, I had to reverse engineer the API (which would make a great blog post) that the Wifi Smart plug my A/C unit is plugged into uses. If I could get things working inside of Homekit then I wouldn’t have to depend on their private & proprietary API not changing.

I would also like to have my Ruby code check more info that just the inside/outside temperature. I’d like it to also look at the UV index, as that can have quite an impact on the temperature indoors. If I got the temperature shoved into Homebridge, and the Smart Plug in there as well, it’d be a lot more manageable code-wise to add another dimension to my K-Nearest Neighbors that I’m using to decide if I want A/C or not.

I was reading a machine learning book when I wrote the code and wanted an excuse to implement something. So the “Should the A/C be on” question is answered by using K-Nearest Neighbors based on past times I’ve specified that the A/C should be on.

So I want to add the UV index to that, and probably a look-ahead to the next hour on the forecast. Currently when the day is warming up the A/C will kick on and off until the middle part of the day at which point the A/C kicks on and the room is too hot for it to cool down. I’d like it to go ahead and notice that it’s going to be too warm later on.

So that’s:

  • move sensor readings to Homekit via Homebridge
  • add UV index to my sensor readings
  • add UV index to my decision matrix
  • add forecast look-ahead to the decision tree

I’ll need to look in to weather the look-ahead would work well as just adding it as a dimension or if I should just perform the calculation again with the forecast temperature. I kind of like the latter.

455 Words

Evented Programming in Ruby

So most people when they think of Evented programming in Ruby, their mind immediately jumps to EventMachine, or HTTP requests, or maybe if they’re really a nerd SAX processing XML documents.

EventMachine really stole the show when it came to the scene and to my knowledge just became the de facto for anything event based. When Node came around the scene seemed to change to “if you want to do use event loops use JavaScript”.

But those are far from the only places that events are useful in Ruby.

One of my favorite lessons taken from Elixir was the solid emphasis on sending messages. In Elixir, it’s a requirement that you send messages to talk between processes. It’s the only way for one process to talk to another. By requiring processes to communicate through messages, it repeatedly hammers the idea of clearly defined interfaces between processes into the forefront of the developers mind.

Elixir does all this because Erlang’s lightweight processes and functional nature make processes the way to maintain state and organize your application.

The fact is though that all of the best lessons from organizing code using processes and messages are also really awesome when applied to Object-Oriented code in Ruby. Thinking of “calling functions” as “sending messages” is an often touted idea among the best OO devs, so this isn’t anything new. There’s a reason for it. Building code by designing interfaces brings the same benefits to any OO paradigm that Elixir takes advantage of because of the rules forced onto it by Erlang.

In Objective-C, Cocoa makes heavy use of defined interfaces and delegate objects. Throughout building a Cocoa app you will create several objects and set a delegate property that will be used to process callbacks and customize behavior or the object.

Relying on this requires having a well defined set of public interfaces that delegates can implement and an equal set of notifications that are sent by objects.

This paradigm shifts the focus from “calling” functions that manipulate state to sending and receiving messages that react to events in the system.

Shifting this focus places a greater emphasis on building well documented, flexible APIs. Inside my own code, if I can make a system that has a defined interface to maintain whatever state in necessary, I’ve won.

Living this world allows you to leverage events that are already happening in your system. Currently Rails makes use of this in a handful of ways. For instance, the different gems that Rails provides use ActiveSupport.on_load to take care of initial setup tasks like adding configuration options that each one provides. When you require "active_record", it adds an on_load event listener that gets triggered when some of the base classes get loaded. This makes it possible to both defer configuration and take advantage of eager-loading.

Rails also uses as separate implementation of this in its fantastic instrumentation setup. Rails uses its instrumenters internally to generate most of the awesome log statements you get in development. All of those awesome “Rendered User#index” with timestamps for how long was spend in controllers/views/sql all use this sort of internal pub/sub. In ActiveSupport::Notification there are examples of how to hook into the default events as well as how to publish your own. It’s worth a read.

Taking advantage of this in my own code means that I’ve had to put enough thought into it that I’ve picked out the events that I need and provided some appropriate interfaces for interrogating the data that I have.

Since I can’t really explain these things without example code, here’s a project where I took advantage of this.

I was writing some code that needed to check the temperature at an interval as well as respond to bluetooth button presses and POSTs over HTTP. I also wanted to provide access to the state of the system so that I could check and make sure it’s running correctly. I also needed to keep track of the state of an external system (a window unit A/C).

There’s a gem called EventBus by Kevin Rutherford that allows you to register objects to receive events and provides an interface to publish events to them.

To do this I ended up with three events:

  • tick: for triggering the periodic temperature check.
  • toggle: for triggering a change of state.
  • state_changed: for tracking the change of state. These pass a to argument.

The objects I ended up with were:

  • StateManager, which subscribed to state_change events to maintain a record of state (on/off).
  • TemperatureManager, which responds to tick. If a tick is received, it checks the temperature and will publish a toggle message the temperature is too high and the A/C needs to be turned on.
  • A Sinatra app that publishes a page. If a POST /toggle is received it publishes a toggle message.
  • an AirConditioner object that responds to toggle messages and turns on and off the A/C. If it’s successful in changing it, it publishes a state_changed message that StateManager will get.

In the end, we have a handful of events and objects all with clear responsibilities and interfaces. Each object is then free to handle those events as they come, or not depending on how that object decides.

Having those events defined also made it easy to add functionality. I went out of town and wanted to have it send me push notifications when my A/C was turned on. I already had a defined place to add that. I simply added another object, a StateChangeNotification that subscribed and responded to a state_changed message. Without changing any of the code that actually made the app function I was able to add a whole new feature.

The most obvious critique is that this could also be done with just OO, and that’s 100% valid. But by setting these constraints I force myself to thoroughly explore & design the objects that I end up creating. And in a language that can really encourage just throwing state around places.

So, this is a somewhat off the wall example. Most of the time I’m not writing code to control my air conditioner.

Most growing Rails applications make use of Rails’ ActiveJob to defer processing of taxing tasks. ActiveJob, and the older Resque or Sidekiq or whatever, are basically a defined way of sending messages without being explicit about sending messages. To an large extent, it’s a bit of an unhelpful abstraction, because for the large part the way I’ve seen it used forces a paradigm that could be useful internally in a process to only be used for inter-process jobs.

I don’t really have an end for this. I could keep rambling on about how amazing message passing is, but this is already entirely too long and needs to stop.

1,141 Words

Using Object Models over Service Objects

This week at work I ended up having a conversation that I’ve had before about when to use service objects vs. when to use PORO model classes.

I’ve had this conversation before, a few times, and vaguely recalled that last time I was on the other side of it. So I reached out to my friend Scott hoping he could set me straight and he did. I’m going to go ahead and write it up so that hopefully next time I’m in this situation I can just refer back to here.

So, what the fuck am I talking about?

The general consensus around the office was that Service Objects were a fad that flew around the Ruby community sometime around 2014. At the time, I loved them.

Essentially, they provided a place to house “glue code” that you were going to use multiple times, usually stuff that was fairly business logic-y, or stuff that was complex and didn’t quite belong in a model or just in a controller action.

With a definition that vague, how could anything go wrong, right?

So I ended up using them a lot. I used them for things like importers. I would have a beautiful PersonImporter class that would handle things like creating a person with a given set of params. I saw the benefit because this application was creating people both in the controller and in a handful of other places like rake tasks that imported records from other sources. At this time this project also had an “evented model” where different services could talk to each other by publishing events, and some of those events might cause a Person to get created, and so it was great to have a single place that handled translating params, creating a person, validating it, creating related people records (which might involve fetching additional information), etc.

So I liked them. I thought they were a dream.

Essentially, the paradigm it had me adopting was something vaguely bigger and less defined than MVC. I had controllers, which were essentially one adapter to access my service objects (rake tasks and event handlers being two others). My models were strictly related to pulling info from the DB, validating individual record values, and defining relationships between themselves. My views were Collection+JSON serialized using the Conglomerate gem.

A little while into living in this dream world, I went to RailsConf and watched Sandi Metz’s talk about “Nothing is Something” , and like so many others I was wholly inspired to write better, more focused, more object-oriented code. If you haven’t watched that talk, seriously quit reading this and go watch that talk. You won’t even need to come back and finish this blog post because you’ll already know.

I couldn’t figure it out on my own, so I got Scott to sit down and watch the video of that talk. Here is, as far as I can remember, what we came up with.

Essentially, we were using Service Objects to hide procedural code inside our Object Oriented design. Mostly to avoid coming up with the correct nouns. Fucking naming things, right?

I didn’t know how to name an object that imported things outside of verbing it, so I just verbed it and threw it in app/services. Which, like, totally fine. It’s a cop-out, but, seriously fuck naming things.

The problem is that it encourages you to write less object-oriented, more procedural style code.  I had a lot of code that looked like this:

https://gist.github.com/jakewilkins/92816efbae675cc3a739583a5703cef0

Which is at least organized. It’s not great, but it’s pretty easy to see how I got here. It’s stuffing the complexity further and further down, hopefully creating a top level that’s straightforward and easy to follow.

The thing is, it would be fairly trivial to take this and make it more traditionally OO.

If we just name it correctly, this same thing can happen and be nicely wrapped in much more familiar OO mindset.

Essentially, my service objects were badly formed wrappers around an object that represented some sort of ExternalModel that I didn’t have named in my app.

To name these models better, let’s have a for instance that I’m important people from IMDB using my PersonImporter.

I could instead have a Imdb::Person, living inside of app/models/imdb/person.rb. In IMDB, people have multiple movies, and I would want to suck those down to. So I could have a Imdb::Movie model stored similarly. When a Imdb::Person needs a movie, it creates and instance of a Imdb::Movie, or vice versa.

Once we have our objects setup, sending the familiar #save message would handle translating those external models into their equivalent internal counterparts.

The benefits here seem kind of small. We definitely haven’t solved all the worlds problems.

But I think there is absolutely a benefit here. We’ve avoided introducing a poorly defined abstraction that we have to deal with for the lifetime of our app. Having that model named correctly clearly defines what it represents. It should be clear to anyone looking that a Imdb::Person represents a person, who has something to do with IMDB. I go back and forth in my head whether #save should be #import. If I figure it out I’ll try to come back to add it here so I don’t forget again.

I think for me, service objects were a necessary stepping stone to get from spaghetti everywhere to something more OO. They did a fine job of centrally locating logic that would otherwise have been spread around my code, some in controllers, some in models, and all leaking bugs.

But ultimately I hope next time I remember a little quicker that naming things correctly is always a good idea and in the end leads to cleaner, clearer abstraction layers.

974 Words

Growing Elixir Applications

Elixir is an awesome new language with some amazing capabilities. It brings with it the power of an amazing framework in OTP, which has been built and used over the better part of two decades.

On top of this, new frameworks have already been built and are maturing nicely. Pheonix, the web framework, is already an amazing front runner that makes it easy to build an incredibly performant website.

Ecto adopts the Repo pattern for database access and already has great support for a growing number of databases. Postgres has first class support for some of its newest and favorite features. It provides easy wrappers for relations of many types as well as support for using Postgres extensions like jsonb.

All these things make it easy to start a new project in Elixir and get it up and running quickly and simply. Spinning up a new blog, a new chat app with live updates over websockets, or an online store becomes as easy as it is in languages that have been around for years.

A quick aside for the less familiar. Elixir not only comes with easy & free interoperability for any Erlang based library, it has a robust dependency management system called Hex. Hex is like RubyGems and Bundler all rolled up if you’re coming from the Ruby world. It allows you to specify dependencies in a file and does version resolution to make things work. It sticks with a more strict `locking` strategy, where gem versions are hard locked into a lock file. But I’ve digressed.

Elixir already provides most of what developers of much older languages are used to. I struggle not to call them more “mature”, because in my view Elixir is mature because of its toolchain.

As a developer, that toolchain makes all the difference. For starters:

  • repeatable builds based on locked dependencies
  • stable, pluggable, easily accessible REPL for trying things out
  • generators for new projects with:
  • projects like Phoenix can provide their own templates for generators of their own
  • prebuilt deployment workflows (some battle proven for years inside Erlang)

All these things that just waste developer time ordinarily are handled. Every hour I spend time debugging an app that worked just fine last time I worked on it is an hour I’ve wasted. Having a prebuilt toolchain allows me to skip that shit and get with building new features.

So, basically, Elixir is magic and it just makes everything perfect and easy and perfect and it’s a magical world right?

Not quite.

Elixir and the Erlang zero-cost pass-trhough means that we have to play by the majority of Erlangs rules.

Erlang is a functional language, and variables are immutable unless explicitly freed.

So we’re functional, not object oriented. Which means that variables aren’t objects and so we can’t have state that mutates.

Which, as we’ll see, is actually pretty fucking awesome.

What all this means is that it will force you explicitly manage state that you need. See, when I came to Elixir from Ruby what I realized is that I was using a fuckload of hidden state. I was using objects to store state without actually thinking about why I needed that state, or how to access it.

By making all of that handling explicit, and it impossible for someone to alter state in a way that you don’t want, a developer has to design her state.

After working this way for a while, I found that when writing Ruby most of my bugs (and usually the most tricky ones to solve) were found in mismanaging my objects state. Something was changing state where I didn’t expect, or some state was getting duplicated instead of altered in a way that it wasn’t falling through.

Quite simply, that can’t happen in Elixir. Variables are immutable, which by definition means that no one is reaching in and fucking with your shit. State is handled explicitly, so when I update it, I know. I have to do it purposefully. Not only that, I have to write code that allows me to manipulate that state.

All this makes for fewer bugs, fewer hours spent debugging, more time building features.

Oh, and how can I forget. As a compiled language, a majority of my stupid typo mistakes get caught at compile time. Now, I know that a lot of people, myself included struggle hard getting used to a compiler. They spend time “fighting” it and think it’s time wasted.

Here’s the thing though, that code wouldn’t have ran in a non-compiled language. It would have failed at runtime, hopefully getting caught in tests, at which time it would have blown up anyway.

I like to not only think of the compiler as my friend but as a baseline level of tests that I get for free. You still need to write the appropriate tests and make sure that going forward you’re doing all the things you need to do, but I don’t need to waste a bunch of boiler tests making sure that my interfaces are adhered to. I get those for free when I get my code to compile.

I feel the need to include several paragraphs of what I’m not saying in regards to test but I’m going to give y’all the benefit of the doubt that you know that I’m still saying you should write tests when they make sense.

Many, that’s a lot of crazy cool shit that you pretty much just get for free by using an awesome language. Congrats for making a good choice!

But, it is still a new language. It provides all the correct flex points for building incredibly powerful software, maintainable, and efficient software.

However, its newness limits the documentation available as well as the guidance provided for growing this software in a way that takes full advantage of all of these characteristics.

That’s what I aim to change. I hope to put together a book and several parts to a series that document best practices for growing an Elixir project past the usefulness of the generators and currently available blog posts.

1,013 Words