Even though the game's resolution is super low and there's no obviously fancy surface shaders going on, the geometry/object count is pretty high. Normally that wouldn't be a problem but since the 1-bit rendering technique requires two passes, sending all the geometry to the GPU twice eats up a lot of frame time. Yesterday I decided to sit down and see if I could get all the rendering done in a single pass (plus post-processing).
Are you changing the mesh geometry each frame? That's the only way you'd be sending a lot of stuff to the GPU. Mesh data resides on the GPU's memory after it's allocated by the driver. I'd say it's likely the slowdown was because of Unity dynamic batching breaking down due to your multi-pass shader, resulting in excessive draw calls.
Multi-pass shaders will break batching. Almost all unity shaders supports several lights in forward rendering, effectively doing additional pass for them. The draw calls for “additional per-pixel lights” will not be batched.
http://docs.unity3d.com/Manual/DrawCallBatching.html