Really depends on the rest of your infra and what your game is like.
Ask yourself how many states you need to manage, and how deeply. If it's just a handful, state machine is probably overkill. If it's dozens with complex transitions, it might be worth it.
Ask yourself whether you want your states to draw themselves, or have some sort of master renderer that knows how to interpret each state.
I had decent success in Chromathud with a simple stack of states that knew when to switch to other states and how to draw themselves. Had a splash screen, start/options menu, several game modes and a pause menu, with fade animations. Found it easy to add new states as needed, no unwanted crosstalk. C#/XNA but same ideas apply. I think if I had needed any more complexity, for example, switching directly between game modes, this would have gotten hairy.
https://github.com/Cheezmeister/Chromathud/blob/master/Chromathud/Chromathud/FacetManager.cs