Presentation Layer MVC explained differently

The Model-View-Controller (MVC) pattern has been around a long time.  One thing I've noticed is that recent adoptions of the words MVC by the web community have blurred the original meaning.  Models have  become something that live only on the server-side, which causes one not to apply the design pattern for the client-side.

The wikipedia has a good write-up on MVC here.  I've clipped a section:

In addition to dividing the application into three kinds of components, the MVC design defines the interactions between them.^[5]

• A controller can send commands to its associated view to change the view's presentation of the model (e.g., by scrolling through a document). It can also send commands to the model to update the model's state (e.g., editing a document).
• A model notifies its associated views and controllers when there has been a change in its state. This notification allows the views to produce updated output, and the controllers to change the available set of commands. A passive implementation of MVC omits these notifications, because the application does not require them or the software platform does not support them.^[6]
• A view requests from the model the information that it needs to generate an output representation to the user.

To me, things start with the model.  Models exist anywhere there is stateful (mutable) information.  Note that the concept of event notification isn't strictly required - one can have a passive implementation of MVC.

In the above definition (and in the original MVC intent), a controller was anything that sent commands to a model in order to change its state.  It could also tell the view to change the presentation.

Views generate the output representation of a model to the user.  Note that a view can also accept commands from a controller in order to alter the presentation.  The implication is that views aren't required to be stateless.  They have their own internal models!

***

Where things have gotten messy is the attempt to simplistically map MVC onto web applications.  Models live on the server, views live on the client, and controllers are split between the two.  That is mostly okay until one runs into complex user interfaces.

The other large bit of messiness arises from the lack of composition in the original MVC description.  Controllers and views are associated with one and only one model.  There is no guidance on how one structures interactions between MVC components.

In my prior post, I described the composition of several UI components into what is essentially a view.  Search and select are simply navigation aids.  The interactions are complex enough to warrant decomposition into multiple cooperating objects.  Those objects mostly follow the MVC design pattern, but they are doing so in the context of the presentation layer.

***

A few concepts have become more clear to me.  One is to think hard about the models necessary to support the presentation layer.  Model boundaries often result from the need to obtain information asynchronously from the server and the submodel is nothing more than the server response.

Explicitly handle the cases where the asynchronous response is stale with respect to new model state.  In customer navigation example, the user can continue to modify the search criteria while the list of possible matches is being retrieved.

Models process events.  Those events can be commands from a controller, raised by other models, be the response to an AJAX request or a timer event.

One should try to have loop-free event chains between models to avoid infinite loops.  Ideally one does not want model A subscribed to model B and vice-versa.  This isn't always possible but should be aspired to.

MVC components should be standalone.  The event processing paradigm can tend to couple together components.  In the customer navigation example, one could design the "possible customer list" component to ask the search input component for its current criteria.  That makes it harder to isolate those components when testing.

I've found that adding a simpler listener proxy isolates the components.  A listener proxy acts on behalf of a component so that the component itself never registers as a listener.  The proxy listens to the event and then issues a command to the component (e.g. "queue new search criteria").

Using MVC/MVVM/MVB in JavaScript for Rich Application Interfaces.

Rich Applications Interfaces make heavy use of JavaScript to create the desired user experience.  Structuring and organizing that Javascript code is a bit of art and we are now seeing the emergence of "frameworks" such as Backbone.js to assist.  It is somewhat odd that it has taken this long to mature this area.

The underlying issue I think is that the web community started using Javascript for "glitz" rather than rich UI's.  As a result, much of the teachings and tutorials focus on using JavaScript to make your web-page look attractive.  For example, one has all sorts of animations and slideshows, fading effects, etc.  Each of those "glitzy" examples tended to be a local effect and required no coordination with other elements on the same page.

As an example of a rich UI, consider a screen which is used by customer support to locate a particular customer.  Part of the screen consists of "Live Search" where the system is searching for a customer.  As the representative types, AJAX queries are sent to the backend server to obtain a list of possible candidates.  The representative selects one customer from the possible matches and expects the screen to display enough information to verify they are working with correct customer.

In this example, we have multiple coordinated entities on the screen.  These are the user input for the search, the possible customer list, along with a detailed view of the selected customer.  Let's visit each entity and describe how it is expected to work.

Search Input

The search input is a text input with validation.  As the user types, the input is validated and a possible auto-complete suggested.

One thing that is important in the UI is that user input is not blocked waiting for AJAX responses.  So our input cannot spin-wait for a response.  It also has to account for the AJAX call being asynchronous and possibly having a large latency.

This suggests a simple MVC for the user input.  The user input keeps tracks of what the user types and provides validation.

There is a model in this UI component which contains what the user typed and its validity.  It generates events when the underlying model changes.

Possible Customer List MVC

The possible customer list component could be thought of as a service (or an actor) whose job is to retrieve a list according to the current search criteria. It might generate an AJAX request each time the search input changes.  

This component has a model which is comprised of:

• A (conceptual) queue of input search criteria
• The result set
• A flag indicating an outstanding AJAX request

In MVC terms, control over the model happens by pushing a search criteria to the front of the queue.  The result set is what is viewed.

JavaScript programming is inherently single-threaded.  One often shifts to message-driven design concepts in order to capture behavior.  In the case of this component, it responds to two messages (aka events).  One message is the arrival of new search criteria.  The other message is an AJAX response.

The arrival of a new search criteria message results in the following behavior:

• The criteria is pushed onto the queue
• If the queue is not-empty and there is no outstanding AJAX request, send an AJAX request for the search criteria at the head of the queue.
• Mark the result set as invalid.  Notify any listeners if there is a result state change

The AJAX response message processing is:

• Clear the outstanding AJAX request flag
• If the search queue is empty, we know that the search was the most up to date.  Then:
• Update the result set and mark it as valid
• Notify all listeners
• Else:
• Discard the response and leave the result set as invalid
• Drain the queue until the last element is reached
• Submit an AJAX request for the last element

Note that this is a simplified version.  One often uses timers to notice when the user has stopped typing to avoid sending excessive AJAX requests.  Timer expiry is handled as another form of message arrival.

Note as well that this MVC does not touch the user presentation.  It is purely a behind-the-scenes service.

Presentation of the Possible Customer List

This MVC is wrapped around a "select 1 of N" component.  The MVC listens to changes in result set status and updates the underlying component.  It provides a view of the current selection.  It notifies listeners when the current selection changes.

Presentation of the Selected Customer

This is an MVC as well.  It responds to changes in the current selection as well as AJAX responses.  Like the possible customer list, it needs to address changes in the selected customer while it has an outstanding AJAX request.

Next post: aligning this design with published MVC/MVVM/MVB principals.

Models, Controllers and Views

It has taken me a while to get my head around Rails-think for Model-View-Controller. Here are some notes on what I've learned.

Credits:  The testing patterns can be found on the relishapp website.

Rails MVC concept simplified

Web-requests go to a controller.  The controller usually interacts with a model to carry out model-level actions.  Results from those actions are sent to a view.  The view is responsible for returning results in the web-response.

Views

It is very easy to get de-railed (sorry) when writing a view.  HTML templates are filled in using embedded Ruby.  Since it is embedded Ruby, you can write some very expressive code within the view.  Often, one takes a short-cut and places domain-logic in the view code, sometimes without even knowing it.

Views need to stick to the presentation of data and avoid domain logic.  This will become more apparent when one writes tests for views.  If the presentation includes domain logic, you will spend a lot of time writing complex stubs for that domain logic.

My rule-of-thumb is that a view template should only render a single object or a single list of objects.  That is an odd statement as views can support very rich presentations.  The idea is that one should break down a complex presentation into partial views or just partials.  This is to support design-for-test.

As an example, consider a view which presents both a Widget and a Gadget.  That view should have a partial for rendering the widget and a partial for rendering the gadget.  One then tests each partial outside of the main view.

Note:  Even when partials render a single object, one should build a stubbed version of the object in your test code. Otherwise, domain-logic correctness creeps back into the test code.  Stubbing is easy when one is filling in attributes.  It is slightly more tricky when stubbing out an actual method.  One uses the Klass.any_instance.stub pattern.  That pattern is described here.

When testing the main view, one should stub out the rendering of the widget and gadget views.  The relish people provide a good description here.  The technique use stub_template to replace the rendering of a partial template with a stub.  Quite often, the stub merely renders a distinctive string like "Gadget #{@gadget} goes here" where one stubs off the @gadget instance with a string.

Note:  I have to finish writing about models and controllers.

Models

Models are supposed to represent a portion of your overall information model.  For both good and bad, Rails ActiveRecord models are at the persistence layer.  That layer is also known as object-relation mapping (ORM) layer.

Controller MicroArchitecture: Functional Authorization

One common use case for functional authorization is pretty simple:  You can't do function X unless you are signed in.  As I've mentioned in other blogs, I had been following Michael Hartl's tutorial design patterns which included authorization.

Hartl located common authorization functions in the SessionsHelper module.  Sessions being an overused term, it took me a while to understand that there is a blend of Authentication and Authorization in that module.

Authentication caching is handled via cookies and is in these methods:

• sign_in
• sign_out
• current_user
• current_user=

Functional authorization support is in:

• signed_in?
• signed_in_user
• store_location
• redirect_back_or

Data authorization support is supplied via:

• current_user? which compares the current user against a different user (often the owner of certain data)

In each controller, one added the before_action :signed_in_user filter.

A Variation

The Ruby-on-Rails/ActionController overview (section 8) discusses moving the check into the ApplicationController.  For those controllers that don't need signin, one opts out by using :skip_before_action.

Observations

In both approaches, Authorization Policy is distributed to the controllers.  The second design pattern is a little less so but one still needs to check each controller to see if there are any exemptions.

I've also observed both an increase in test complexity and test run-time when authorization is enabled.  Creation of a valid user and sign-in before each test take time.  One also needs to test each method against non-authorized users.

Formalizing Authorization

Like the variant above, I place a filter in my ApplicationController

include AuthorizationHelper ... before_filter do |controller| authorize( controller) end

The ApplicationHelper module:

module AuthorizationHelper def authorize( controller) require_signin = true if( controller.is_a?( MyExemptController) ) then require_signin = false end if( require_signin && !signed_in?) logger.info "Signin required for the #{controller.class}::#{controller.action_name} action" store_location flash[:notice] = "Please sign in." redirect_to signin_url end end end

In this approach, all controllers are unaware of the authentication policy. This allows one to look in one place and see what the intended policy is.

Note that the code makes use of the undocumented .action_name attribute. This can allow per-method control over authorization.

Controller Micro-Architecture

A Controller supplies one or more web-service API's.  Those API's are most often closely related to a single Model.  For example, one has a Widget model and a WidgetsController.  That close relationship can cause the lines of responsibility to become blurred.

Controllers also coordinate with the presentation of the response.  Again, the lines of responsibility can be a bit a fuzzy - does the logic go in the controller or in the .erb file?

Here is my sense for controllers:

• Functional Authorization
• Does the user have to be signed-in to access the controller's method?
• Is the user allowed to access the method?  For example, is it superuser-only?
• Check for basic argument correctness
• Required parameters
• Extraneous parameters (e.g. white-list)
• Data Authorization
• Is the user allowed to see/modify/delete that particular data?
• Invocation of domain logic
• Error handling
• Assigning domain logic results to instance variables
• For use by result rendering (HTML, XML, JSON)
• Selection and invocation of the result rendering code
• Often done implicitly by Rails!
• Correct presentation of errors

Next:  Best (for some relative value of best) Practices for each area of responsibility.

RESTful Design Part 1

There is a lot written on RESTful design. IMO, most of hides the purpose of the RESTful design pattern. I think it worthwhile to restate "why" before looking at the patterns.  In web-applications, the client and server each have state.  Due to several causes, the client state and server state can lose synchronization and become inconsistent.   Some of those causes are:

• Communication failures and timeouts
• The ability to move off a page and then back
• Bookmarked pages
• The refresh button
• The infamous back-button

These causes can result in the equivalent of a partitioned system between client and server.  The CAP theorem states that there is a trade-off between Consistency, Availability, and Partition tolerance.  In all web-applications, the choice has been made for Availability and Partition tolerance over Consistency.  The availability choice may not be obvious; it is a result of choosing not to indefinitely lock up the user's web browser.

Fielding's thesis on RESTful design proposes a design pattern to minimize the potential inconsistency.  Quite simply, try to make the server as stateless as possible. If the server has no state, there is nothing to be for the client to be inconsistent with.

Fielding recognized that the application server is responsible calculating the "next state" (in most cases).  He relaxed the server-stateless constraint by allowing the state to move from client to server and back again.  Hence there is State Transfer in REST.

That state transfer from server back to client can be found in the HTML template process.  A template is often filled in with content and links specific to a state.  Another good example is the addition of hidden elements within a form.  Those hidden elements provide additional context (or state) to the server.

Generally, an application is RESTful if the server retains no session state between requests. There are some places where keeping session state is useful. For example, keeping authentication (signed-in) state is very useful.

***

But what about RESTful API's?

Those are RESTful on a small scale and are part of making an application RESTful.  But just providing RESTful API's does't mean that the application is RESTful.

***

Controller Design Patterns

Rails has a concept of a controller. Simplistically, a controller is something which responds to web requests. I've seen several different controller patterns emerge in my applications. These are:

• Resource (or asset) service.  I use this pattern when I need to return an image or other monolithic object without exposing a directory path in the URL.
• JSON-based restful API's.
• HTML document services.

The JSON patterns are used to provide access to the business and/or domain logic.  These tend to be fairly pure and understandable.

The HTML document services are a hybrid of presentation, business, and domain logic.  They can be messy as the presentation is coupled to the business part.

For both JSON and HTML patterns, there are variants:

• ActiveRecord model wrappers.  The ActiveRecord model pattern supplies well-checked (robust) CRUD operations.  Constructor parameters use the params associative hash and thus need to handle checking and deserialization.  The controller code delegates all of its work to the underlying model.
• ActiveModel model wrappers.  These are akin to ActiveRecord, only not persisted to the database.
• Other

Rails is really good at generating controllers that are HTML ActiveRecord model wrappers.

The HTML controller has another variant which is composite presentation.  Parts of the composite happen naturally in Rails using application templates.  Other times the controller is merging the presentation of multiple other controllers.  This happens on things like user account summary pages.

Subsequent blogs will talk about controller design problems and solutions.

Architectural Drivers

My apologies to the reader if this sounds obvious.

Application software is now delivered over the Internet.  An extreme type of software delivery is the web-application. One points a browser at a web-site, Javascript is downloaded as it is needed and the application runs.  This type of delivery is quite popular for the following reasons:

• The software supplier is dealing with a single version of the code.  Bug fixes and enhancements are immediately available to all customers.  Support is greatly simplified.
• The software is not stored on the user's computer. You have to access the server each time you use it.  This is basically what enables the entire SaaS business model.
• There aren't that many client machine architectures, i.e. browsers.  It may be a bit ugly but it is not as bad as needing to compile a new binary for each machine.  Technology is not getting in the way of addressable market.

Application delivery times are pretty much a function of the application size and bandwidth.  Somewhat interesting, mobile applications operate in world where repeated downloads of large applications are both slow and expensive.  One installs a client application on a mobile device to minimize both application start time as well as the cost of bandwidth.

The ramification is that what is valued in a web-application is different from what is valued in a mobile application.  This results in fundamentally different technical approaches.  Someday they may be harmonized but that day isn't today.

Since a web-application is download-when-needed, the part that is downloaded to the client is mostly concerned with the user interface.  Server response times (latency) aren't fast enough to run highly interactive UI's over a network.  That work gets done in the client.

Some consequences:

• People try to minimize the amount of code downloaded.  You'll see people publish stats on the size of their Javascript library as the size impacts page-load times.
• One minimizes the amount code downloaded by not putting things there you don't immediately need
• Focus on UI, leave business and domain logic on the server
• Within the UI, focus on interactive UI and leave static presentation on the server.
• Javascript libraries become page-specific.  Yes, the wheel of reincarnation has spun and we are managing our own overlays.
• The server has multiple roles and functions.  It not only has to deal with business and domain logic, it is also implementing portions of the presentation layer.  The server portion of the presentation layer is typically implemented using a templating system.

Most of the development frameworks out there miss this fundamental truth.  Languages like PHP are great at templating but don't help at all with Javascript.  It seems that other frameworks (e.g. Rails, Microsoft's Razor) try too hard to use "just one programming language" and neglect highly-interactive UI's as those UI"s need Javascript.

Informal History

I really became aware of applications with GUI's when I first used one of the original Apple Lisa's.  In the early 80's, I was lucky enough to work at a company that had one and I got to use it. A little while later the Lisa was followed by the Macintosh.  Apple allowed people to write code for the Mac and there was a wonderful series of 3 books named "Inside Macintosh" that explained how things worked.

The first Mac's had no memory management unit, so everything used Handles to allow storage to be reclaimed.  And with no MMU, threading was a programming convention.  Just like Javascript today, the application was single-threaded and used an event loop.

But what was really fascinating was the conceptualization and clarity of the UI component framework.  Menus, windows, labels, buttons, etc. were all there.  I'm not sure if Apple was the first to do things this way but they were the first to teach it to me.

30 years have gone by since then but not much has changed in the approach to UI frameworks over those years.  We've had Windows Forms, X-Windows and Java Swing in that period as well.  All have much in common with the framework organization found in the Mac.

Lately we've heard about thin-clients, web-apps, mobile apps, etc.  X-Windows was designed for thin clients.  People even sold X-windows terminals.

Anyone remember Java WebStart?  It was a way of downloading and launching a thick-client Java application from a web-site.

The web has evolved but it never really had a true Web 2.0 version.  Changes were done incrementally.  The key non-change is that what is shown on the screen is what is in the DOM.

What has struck me about today's web application technology is that how much has remained the same with a few critical differences:

• The thin-client now runs in a web-browser and its native graphics abilities are relatively awful
• Server-side processing (what used to be done in worker threads) cannot easily notify the presentation layer that the model has changed.  The client has pull and poll to get anywhere
• Bandwidth and communication latency are important.  Bandwidth is somewhat related to total information transferred (cost factor for mobile devices).

In my next blogs, I'll look at how these differences are shaping our current architectures.

Introduction

I'm interested in Web Application Architecture as it is a technically challenging subject which entirely subjective.  There are a lot of different answers, depending on what you'd like to do.

When I'm reading the literature or looking on the web, I don't see very many good discussions on this topic.  So I'm taking a run at this topic by starting this blog.

There is a lot of work going on.  Ruby-on-Rails, Microsoft's MVC, PHP, etc.  All seem to address a part of the problem but don't address the larger design problem as their tools are not as broad as one would like.