Rails error - "marshal data too short"

This one gave us fits for days because we were troubleshooting in the wrong manner.  This error generally is not caught by a Rails application because it happens as data is transported between the data storage layer (in our case MySQL) and Rails itself.  So its outside the scope of your application error handling.  This results in a rough experience for users of a web application who see nothing but a white screen with un-style text "Application error (Rails)".  Dead end for them!

What causes the error?

Generally, it happens when session data exceeds the storage capacity of the database field storing it.  It's truncated on insert/update and then when it's marshaled back, Rails doesn't know how to read it because vital parts have been severed.

What can you do about it?

The lazy fix is to use a larger field type for storing data, say blob or longtext.  But that eats database overhead and doesn't really address the root cause.  A better fix is two fold:

1. Be aware of this issue so you can better manage large session data, and
2. Understand implications of marshaling as it applies to the Confused Deputy problem

The Confused Deputy Problem

Here's a decent background at Wikipedia, of all places.  In short, marshaling data is a way to make sure an application is not vulnerable to becoming a confused deputy.  You just got to make sure it doesn't kill itself in the process of avoiding confusion.

Blogged with the Flock Browser