Cool idea, man.
I think you can remove some of the annoyance factor by not actually pausing the game for the screens, but just overlaying the screen as the game progresses normally. Right now it's too much like freezing the game when you try to do anything and it does get very annoying very fast. Though, I'm not sure how well it would work if you're actually doing something and can't see it.
i think it will more annoying and unfair, if you land near an enemy for example, also it will lose the fact that they actually paused everything in mute movies