Monday, October 19, 2015

Subscription Redux

ElixirFBP Subscription

An ElixirFBP Subscription is the "glue" that ties together all of the components in a flow graph. While Components could certainly send messages directly to each other, I found a compelling reason for developing a Subscription as a result of reading about Reactive Streams. After some fits and starts, I have developed a Subscription which seems to be able to run in either a "pull" or "push" mode. This code has a slight smell - I might be trying to do too much with this one module. It may make more sense to have two different Subscriptions - for push and pull mode, or, alternatively, develop a macro that generates a different Subscription depending on the run mode value.

In pull mode a flow typically begins with the last component(s) requesting some number of information packets. This request propagates upstream through the flow graph until eventually reaching component(s) that can provide data. Running in pull mode also allows a component (actually the subscription) to apply "back pressure" to the upstream components. Finally, a component won't respond to a request until it has all the values it needs available at its in ports.

In push mode a flow begins with the arrival of information packets at - usually - the upstream components.  When a component has received all of the data that it requires, it computes a result and sends it to its out port.

If a component is meant to work in both a push and pull mode then its listening loop must account for different sets of messages. For example, here's an Add component:

defmodule Math.Add do
  @moduledoc """
  This module describes an FBP Component: Math.Add
  It is can operate in both a push and pull mode
  @behaviour ElixirFBP.Behaviour

  def description, do: "Add two integers"
  def inports, do: [addend: :integer, augend: :integer]
  def outports, do: [sum: :integer]

  def loop(inports, outports) do
    %{:augend => augend, :addend => addend} = inports
    receive do
      {:addend, value} when is_number(augend)->
        send(outports[:sum], {:sum, value + augend})
        inports = %{inports | :addend => nil, :augend => nil}
        loop(inports, outports)
      {:addend, value} ->
        inports = %{inports | :addend => value}
        loop(inports, outports)
      {:augend, value} when is_number(addend) ->
        send(outports[:sum], {:sum, addend + value})
        inports = %{inports | :addend => nil, :augend => nil}
        loop(inports, outports)
      {:augend, value} ->
        inports = %{inports | :augend => value}
        loop(inports, outports)
      :sum when is_number(addend) and is_number(augend) ->
        send(outports[:sum], {:sum, addend + augend})
        inports = %{inports | :addend => nil, :augend => nil}
        loop(inports, outports)

The last receive clause is used when the flow is operating in pull mode. This message (:sum) originates in the subscription that connects this component's out port (:sum) with the in port to another component. Note that the destination of the send to the :sum out port is a Subscription
process; these processes are all started when a ElixirFBP.Network.start function is executed.

As I mentioned earlier in this blog, I would like to eventually develop an Elixir macro that would generate all these receive clauses based on a more declarative description of the component's in and out ports and computation.

A Refactoring

I refactored my ElixirFBP project - removing all of the Runtime support for the Noflo-ui network protocol. I will develop another project that uses ElixirFBP as a service and re-implement support for this network protocol. This essentially means using Cowboy to manage websocket requests and turn them into calls to the ElixirFBP gen servers. The newly refactored project code is available at my GitHub repository.


I watched José Valim's keynote talk at the recently completed Elixir Conf 2015 (US). I learned that we might expect a new gen behaviour named GenRouter to appear in Elixir 1.3. Early code is available on Github. At first glance, it would appear that GenRouter might server very nicely as a base for FBP. I will follow the design and development closely and, at some point, see whether I can move ElixirFBP on top of it.