next »

[FrankForce]

Hey, I'm working on a game engine and I started looking at frame delta fluctuations and how to best correct for them. I tried posting about this in a few other places and still have yet to find a single person that really understands the issue at all. It's like everyone is in denial that the issue actually exists! Maybe it's just the knee jerk reaction of programmers to think they already know everything and people think I just fell off the back of a turnip truck. Anyway I just finished up my second article which describes a thought experiment to help explain what frame delta variance is and why it is absolutely necessary to correct for.

http://frankforce.com/?p=2636 - Time Delta Smoothing
http://frankforce.com/?p=3522 - A Rendering Thought Experiment

How do you smooth out the delta time fluctuations in your game engine? For those who already understand the situation very well, does my thought experiment adequately describe the issue or what can I change to make it better?

[nospoon]

Hmm...
I didn't really care about it ever, but I think, that the easiest way would be just limiting the delta.
When for example we make the delta to be between 1/20 and 1/60, then we won't really have as many 'jumps' in the delta timing.

And as for smoothing - it might be a good idea to use interpolation, kinda like multiplayer entity smoothing.

I also think, that you should limit your framerate anyway. It might fix the issue partially.
I mean - who needs a game that runs on 1 000 fps.

[FrankForce]

Capping your max delta will not fix this! Also you should definitely not cap your min delta, that is just throwing away important timing information. Locking the delta to always be equal to the vsync interval can help fix the issue, but it causes some problems of it's own.

I agree you should use interpolation, but that is a totally different and completely unrelated issue! Also, that only comes into play when you are using a fixed time step. But none of that will work properly if the deltas coming in are fluctuating all over the place.

Here's a graph of what deltas typically look like that are fed into a windows game with triple buffering.  Keep in mind this has nothing to do with the game itself, this graph could be for any game or really any system/os/hardware...



BTW, I'm not trying to fix the issue myself, my solution works fine for me.  My engine uses a combination of delta smoothing, then fixed time step with interpolation to get the smoothest animation possible.

[Floor is lava]

I use QueryPerformanceCounter for more precise timing and if the time delta is a problem I'll just average the last 3-5 time deltas. 

[ThemsAllTook]

I cringe at the thought of triple buffering and what it does to input latency. Seems like a tradeoff that's not always worth making.

I've never observed anything like that frame time graph in my own games. Since it's a repeating pattern (one long frame every 6 frames), I wonder if you're rendering at an interval that's just slightly off from your refresh rate, so vsync stalls predictably? What I normally do is run game logic at 120Hz and try to render as fast as I can allowing vsync to throttle it, which works out pretty well. I've never done a really in-depth analysis of frame times like what you've done, but I've never noticed enough of a problem to look into it.

[J-Snake]

Variable timestep can always fluctuate. If you want perfect game-mechanics you have to use a fixed timestep. I use my own solution for it. To achieve smooth rendering I use interpolation. Have you looked into it?

[FrankForce]

Floor is lava: It's good to have a high resolution timer, but these fluctuations are not caused by that and will not change as a result of that. You might get some tiny fluctuations from floating point error, that is not this. Keeping a running average of the frame time may help slightly but it is not a complete solution. There will still be stutter with averaging, my solution completely eliminates the stutter caused by delta fluctuations.

ThemsAllTook: If you want smooth animation it's really only a choice between double or triple buffering. Double buffering has 1 frame less latency but you must guarantee that your game can always do and update/render in less then the vsync time or you will skip that frame and it will stutter. This is possible on a console (especially when locked at 30hz) but not as much on a PC where there is more overhead from the OS and it can be run on a range of different systems. Triple buffering affords a whole extra frame of slush time. In my experience 1/60 of a second of input latency is a very small price to pay for perfectly smooth animation.

The graph is not from my own game, it's from a slide by AMD.  It is known as a heart beat stuttering pattern. Here's the full slide...



This type of stuttering is not a huge problem but it is noticeable to me at least and AMD too apparently.  If you save out your deltas to a log maybe they would tell you a different story.

J-Snake - Arg. I'm starting to feel like a broken record but this has nothing to do with using a fixed time step! My engine already has fixed time step with interpolation. You will need smooth your delta whether you use a fixed time step or not. Also, using a fixed time step with interpolation only does much of anything when your game is stepping at a different rate then your monitor.  These days it is much less relevant since newer monitors all use HDTV standards which is 60 fps and PC games typically update at 60.  In the olden days computer monitors had all sorts of odd frequencies. But I do agree fixed time step with interp is the way to go, just that it has nothing to do with this at all.

[J-Snake]

Also, using a fixed time step with interpolation only does much of anything when your game is stepping at a different rate then your monitor.

But you still keep the rate, aren't you? Concept-wise my implementation is perfect. I can set any odd framerate and it will look smooth, even 50fps on a 60fps screen. The only practical problem is that stopwatch sometimes returns extremely low numbers (but never higher than 16 ms) which cause stuttering with interpolation (assuming the deltas are wrong). One might expect it to be be around 16 ms (when 60fps and vsync), but in rare cases it is far off. Fortunately a system restart can help, I am not sure what is wrong on this part yet but it must be something on a system level.

The question I am currently facing on this is (test examples are light-weight, no bottlenecks):

Does vsync always really work that constantly like I assume?-Letting the update always wait until gpu-buffers are swapped. With 60hz screen and vsync enabled that would mean my stopwatch will constantly return something around 16 ms, but it often fluctuates inbetween 8-16. Assuming the stopwatch works correctly it could also be the screen fluctuating in drawings. But I doubt that is the case since the same example would run smoothly in a variable step without delta scaling. So it might be the stopwatch-class (in C#, xna) causing the problem again, or the update doesn't always wait until the buffers are swapped. The problem is that various theories can be valid until the system- level tech is not sufficiently investigated.

[J-Snake]

I have looked into your article now and I have seen you are referring to this "fixed timestep article". The solution presented there motivates one flaw on a conceptual level already, it is the same flaw existing in xna's native fixed timestep. When you update more than once in a row you are prone to get into that ugly stutter-cycle because every time you delay a frame its time will accumulate for a catch up and vice versa. So just replace the while-loop simply with a if-statement and never allow elapsed time getting a higher value than 2*frametime. That will solve it. The only con is that you cannot update faster than the refresh-rate, but practically only monitors below 60hz will be affected by running slower. Most of the people have 60 hz or more monitors so this con is practically non-existent. Also what is important to know is that in a loop with "catch-ups" screen-refresh and updates can never be exactly in sync (except with some artificial tricks). However with one update it natively will for all screens with less to equal the hz-rate you are targeting, if you are doing as told. For example if you are targeting 60fps and the screen is 59.7 the solution will automatically update with 59.7 so no additional frames are going to be accumulated.

Also you want your game to feel responsive, you don't want to use the same input for eventually 2 frames. That is also a motivation why not to update several times in a row. My current game is extremely sportive, you can feel the unsatisfying results of a game-loop with catch ups right there.


I have thought about writing an extensive article about why my fixed timestep solution is superior to what you find in most other articles, including detailed explanation how "catch-up" updates affect stuttering and such.

Check out this game-loop here and tell me how you roll with it:

Code:

 protected override void Update(GameTime gameTime)
 {
    elapsedTime += gameTime.ElapsedGameTime.TotalMilliseconds;         //example:
    if (elapsedTime > 2.0 * frameTime) elapsedTime = 2.0 * frameTime;  //frameTime = 1000.0/60.0

    if(elapsedTime >= frameTime)
    {
        elapsedTime -= frameTime;
                
         // insert your game-logic here
         
    }
      
    transitionAlpha = (float)(elapsedTime / frameTime); //use interpolation in the renderer
 }

[J-Snake]

So if you don't see the same sort of problems in a variable step (no delta scaling), if it runs smooth then it is indeed the bad use of a fixed timestep causing this problems. A regular (with catch-ups) fixed timestep is never perfectly in sync with the screen. That is very important to understand.

[FrankForce]

J-Snake - Looking at your code "if(elapsedTime >= frameTime)" should be "while(elapsedTime >= frameTime)". Maybe you are thinking that in an ideal case it should never update more than once per frame, which is true but your solution is misguided. Sometimes it does happen that you get a very long frame and need to update twice to keep up. Either way this will cause some kind of hitch. The type of hitch your code will cause is everything will slow down to half speed for 1 frame. Using a while loop is better because everything would still move at the same speed but literally just skip a frame. This becomes more apparent when the frame rate itself drops for a significant period of time. Using an if instead of a while will cause slowdown rather then just running at the same speed but a lower frame rate. The "catch-up" code you added is just an attempt to fix the issues caused by not using a while loop, it should not be there at all.

But there is a much more important reason why your code is wrong! Your game engine uses a fixed time step of 60hz. Think about what would happen if you ran your game on a monitor that only supports 30hz. Your game would run at half speed. Using a while loop fixes this because it can do more then 1 update per frame if necessary. This is why while loops are the correct solution. In fact this would be an issue on any refresh rate less then your timestep. It sounds like you are aware of this but don't see it as an issue even though most TVs on the planet still run at 30hz or less.

Also, this isn't very related to the issue of delta time fluctuation and your code does nothing to correct for it!

[ThemsAllTook]

most TVs on the planet still run at 30hz or less.

They do? NTSC is 60Hz, and PAL is 50Hz. Is there another one I don't know about that's more common than those two?

...but yeah, J-Snake's solution is incorrect for running at a constant speed. Allowing your game to simulate more slowly if the computer can't keep up every frame causes lots of trouble, and is completely incompatible with networking.

[FrankForce]

NTSC and PAL are both interlaced, so while NTSC is technically running at 60hz the actual frame rate is half that. If you look on the Wikipedia page for NTSC, the first sentence under the heading "Technical Details"

"NTSC color encoding is used with the System M television signal, which consists of 29.97 interlaced frames of video per second."

[J-Snake]

My solution is best when the refresh rate of the screens is equal or above the intended framerate.
You are confusing the the display rate of a video-playback with the hz of the screen. The minimum hz a regular user might have are 50. You can target that rate if you want to go sure for everyone.
Anyway, that is not the problem.

Programming with catch-ups need a head-room for at least 2 updates in a row, mine does not. Means your solution is also worse for cpu-bound games. If your rendering cannot keep up with the speed I would rather disable vsync than going with a general catch-up solution. Catch-ups are worse for  more consistent/relyable input response. I can explain exactly why but you should work it out yourself.

Whenever there are no bottlenecks and the refresh rate is higher than the target-fps, every catch-up is a flaw, it shouldn't exist. Isn't it exactly the problem you are having with it? What should exist are only delays, the positive thing about it is also that every input is registered within a safe time window.

[ThemsAllTook]

NTSC and PAL are both interlaced, so while NTSC is technically running at 60hz the actual frame rate is half that. If you look on the Wikipedia page for NTSC, the first sentence under the heading "Technical Details"

"NTSC color encoding is used with the System M television signal, which consists of 29.97 interlaced frames of video per second."

Sure, but interlacing halves vertical spatial resolution, not temporal resolution. You're still drawing two separate frames per 30Hz screen update (or 29.97Hz, close enough), just stitched together. I don't know all of the low-level technical details, but in the worst case, you'll have one frame of latency at 60Hz, assuming you have to send the video data for two frames simultaneously. It still works out to 60 unique frames displayed per second.

Catch-ups are worse for  more consistent/relyable input response. I can explain exactly why but you should work it out yourself.

I understand your theory perfectly well, but it's flawed. Slowing down your game to catch up with missed frames may allow you to register unique input for each individual frame, but human reaction time completely obliterates any gain you get from that. If I'm trying to time inputs precisely, getting a few frames where everything slows down is going to throw me off way more than only having 30Hz of input resolution to work with. It's not possible for a human to register that a frame has taken longer to render, recalculate their timing, and press/release a button slightly earlier or later, all within 1/60th of a second.

At the risk of derailing this into the same argument people have with you in every thread you post in, is there a specific situation you can describe in detail where slowing down to catch all inputs will benefit your gameplay?

[J-Snake]

At the risk of derailing this into the same argument people have with you in every thread you post in, is there a specific situation you can describe in detail where slowing down to catch all inputs will benefit your gameplay?

Seems you haven't been much into sportive quality. In this regard catch-ups are only beneficial in a monotonous happening, like when you are swaying your gun in a fps monotonously over a significant amount of time, you assume to keep it up with the time in monotonous scenarios. But games are interactive and most often about change, when you change your movement direction etc. you don't want the next update-frame to act according to the input of the last frame at times.

Let's observe it more closely: Whenever you press a key it is alway at least the duration of a frame so it will register (so delay is no problem)(analog-like inputs can act differently but it still doesn't make a plus on both sides(delays or catch-ups). But whenever you release a key it is nearly an instant moment. Now assume a game is accidently in two very fast update-frames in a row (like it would happen in TrapThem when I would use a catch-up loop). It practically means that the catch-up frame will take your input when you release it in that unfortunate moment. That is not what you wanted. TrapThem is a grid-based game, you cannot stop between the grid-cells. It would mean that at times you are more prone to go one grid-cell further than you should. The time it takes to reach the next grid-cell are 12 frames. In my solution you will always have this amount of time for sure. In a solution with catch-ups you would take up to one frame-time away at times. It is 8% taken away from a safe time-window. And it is something you will feel. I tested all this and the difference is significant. I think people should thoroughly think the mechanical details through when they aim at sportive quality.

Also when your screen is higher, let's say 75 hz and your game is running at not bottlenecked 60fps you can quickly see that there is something wrong whenever you register a catch-up, it should only delay a frame at times since 75 is higher than 60. The only point in catch-ups I see is when your game is gpu-bound and bottlenecking because of that, but I would rather take screen-tearing into account and not sacrifice gameplay-quality. To each their own.

[FrankForce]

My solution is best when the refresh rate of the screens is equal or above the intended framerate.

No, your solution only functions properly at all if the screens refresh rate is higher then your timestep, otherwise it will cause slowdown.  It seems like English is not your first language but you should know that "is best when" is not the same as saying "only works properly when".

If you are really that worried about latency you can run non-vsynced but that is going to look glitchy due to the tearing. The tradeoff of latency for smoothness isn't worth it in my opinion. Running double buffered instead of triple will get you 1 frame less latency but you will need a much more stable update/render without any spikes that I have found is impossible to maintain due to spikes caused by the OS which is why I recommend triple buffering. In my 10+ years in the industry I've found that players automatically compensate for the tiny amount of latency but its hard to ignore stutters and tears.

I'm honestly having a hard time understanding what you are trying to communicate so I'm willing to grant that you could be totally right and maybe I'm just an idiot. The best way to show your ideas is to write up an article somewhere else about it that explains your whole process. I'd really like to hear it, I'm sure you have some interesting ideas.

Sure, but interlacing halves vertical spatial resolution, not temporal resolution. You're still drawing two separate frames per 30Hz screen update (or 29.97Hz, close enough), just stitched together. I don't know all of the low-level technical details, but in the worst case, you'll have one frame of latency at 60Hz, assuming you have to send the video data for two frames simultaneously. It still works out to 60 unique frames displayed per second.

Ok, I hear what you are saying and you are correct. Terminology is important here though, in the case of interlaced frames they are actually called fields. So you could say NTSC runs at 60 fields per second. I'm pretty sure you are right that a game could run at 60 frames per second, treating each field as a separate frame. In the olden days we always just ran at 30 because ps2/xbox were generally too slow to keep a solid 60. It seems like this is still the case with many modern games, for example Starhawk (the last PS3 game I worked) on ran at 30 even for HDTV. Also j-snake's code would still run slower on PAL because it is only 50hz, and there's nothing that can be done about that (except change the if to a while but that's too much work).

[ThemsAllTook]

Let's observe it more closely: Whenever you press a key it is alway at least the duration of a frame so it will register (so delay is no problem)(analog-like inputs can act differently but it still doesn't make a plus on both sides(delays or catch-ups). But whenever you release a key it is nearly an instant moment. Now assume a game is accidently in two very fast update-frames in a row (like it would happen in TrapThem when I would use a catch-up loop). It practically means that the catch-up frame will take your input when you release it in that unfortunate moment. That is not what you wanted. TrapThem is a grid-based game, you cannot stop between the grid-cells. It would mean that at times you are more prone to go one grid-cell further than you should. The time it takes to reach the next grid-cell are 12 frames. In my solution you will always have them. In a solution with catch-ups you would take one frame away at times. And it is something you will feel. I tested all this and the difference is significant. I think people should thoroughly think the mechanical details through when they aim at sportive quality.

Alright, sure, I see where you're coming from. I think I have an alternate solution that gives the best of both worlds: Suppose you have a separate thread for input, which runs independently of your simulation and rendering thread(s). It's able to accept input at 60Hz or more, regardless of your render thread blocking on vsync or other such things. Each input event is given a timestamp when it comes in, and is queued up to be processed by the simulation thread. The simulation reads the timestamps of inputs as it processes them, and uses that to distinguish inputs that changed between two frames, even if it's catching up and processing both of those frames at once. It seems to me like this would allow your simulation to run at a constant speed, allow the renderer to miss a frame here and there if it can't keep up, and keep the high input resolution you desire.

Being able to implement this depends somewhat on capabilities of hardware, OS, development environment, and system APIs, but it should be doable just about anywhere if you either have preemptive multithreading or an input API that gives you timestamps.

Ok, I hear what you are saying and you are correct. Terminology is important here though, in the case of interlaced frames they are actually called fields.

Good to know, I'll keep that in mind for future discussion!

[J-Snake]

Not sure about this, it will certainly be worse for a light-weight game like TrapThem since it is natively in sync with all the happening the way it is. A delay is always a benefit and not a con for the player in Trapthem.

I would carefully look at sync properties in threaded solutions but it might have advantages in some  applications.

[J-Snake]

It seems like English is not your first language but you should know that "is best when" is not the same as saying "only works properly when".

When I say "is best when" I mean it that way. The specification for better is: reacts more consistently to input and better performance for cpu-bound games.

If your spec is different then feel free to deal with it your way.

You have to learn to make a good compromise since you cannot get the best of 2 contradicting worlds. For me it goes like this, why should I satisfy a minority of people using 50hz screens but punish all the rest having 60 hz or more? Ok, in this case I could go safe and make my game 50hz since delays are not punishing in my game for all those with 60+ hz. It is just that I feel safe nearly everyone will enjoy it the way it is intended to be. In worst case those few 50hz- people will get a somewhat slower experience but it is still exactly the same game.

Also keep in mind that wiht a higher refresh-rate like 120+ the related things will natively get more and more consistent.