I'm really interested in this but I have no experience with live coding at all.
me too... (at least not with real live coding tools)
but i've recently tried out Unity, and worked through the introduction tutorials... one thing that appeared to be very useful, was that one can define variables in the scripts, which get immediately integrated into the GUI (like faders, or links to connect objects visually), which can be change
while the game is running (although, they didn't remember modified states somehow...). also a very useful "live dev" feature was, that one can inspect all objects in the game "live", update assets, and make changes to the script code all while the game is running (as far as i remember correctly...).
@c++ dev:
recompiling the whole game for every little change gets sooooo annoying after a while... i can't stand it. it's my main concern about using c++. it's so unproductive and time-wasting... (that just after working on something today, rebuild-test-rebuild-test-.....)
but in Microsoft's Visual Studio it is possible to do some sort of live coding though. just run the program in debug mode, then code changes can be linked in without rebuilding the whole project again (->
edit and continue). for tweaking small things like colors or sizes, that can be very helpful. somehow i didn't use it too often, but it is surely a useful feature in some cases...
for larger changes though, i don't know how far it will keep working correctly, especially when pointers and game objects should keep intact...
in tools like Unity, Game Maker (i guess), Torque (i guess), and high-end engines like the Unreal Engine have various live features built-in, so that development is always much more fluid and faster, and more fun overall (i guess)... using only bare bone coding tools to make games is kind of a torture...

and hindering creativity in many ways.
scripting could be one way to overcome that problem. but for small games it's pretty difficult to implement, especially for "live coding" purpose. reinitialize the whole game state after every change can't be isn't exactly what's called "live", i think... i tested lua/c++ once, and it's not really working out on a small scale, since maintaining the scripting system and all its bindings makes things rather more complicated than easier, in my experience.
so... always interested to hear more about live coding games... interesting topic indeed.