There is already some valuable advice posted in this thread, good job guys. So my contribution will have overlap, but I want to add to it and provide my structured view on the important basics of the subject matter:
First, there are the ingredients and the cooked meal which shape the puzzle: the
mechanics and the
goal:
- The mechanics is the
minimal set of actions from which every action can be composed.
- The goal is the
sequence/s of actions the player is expected to perform (a single sequence satisfies the goal).
Now it follows that you have to juggle the two following demands:
- A change in mechanics, which (in the regular case) changes the set of possible goals.
- A change in the set of goals, which enforces a change in mechanics if the desired goals cannot be satisfied.
But with some experience you notice the following: While a change in mechanics usually introduces a change to the possible goals, a change to the list of selected goals (those you want to provide to the player, for every goal there is a corresponding puzzle) does not necessarily force you to change the mechanics if they are mature enough to express many (interesting) goals. Thus, it is good practice to start both ways, to think about cool mechanics and what you want to do (the goals) with them. But then, once the mechanics have reached some maturity, you focus mainly on the goals you want to provide to the player and only change/adapt the mechanics carefully and rarely, only when the need really arises.
This was some abstract and dry but important talk so far. However, the final question is:
What makes a goal interesting? (in other words: What makes a puzzle great?)
In order to find an answer, recall that a goal is nothing more than a
sequence/s of actions. But the plural option "sequences" is the key here. If the goal is just a single sequence of actions the player has to perform, the puzzle may feel too "rigid". So now comes the question: But when I give the player more options, how do I make sure the intended goal won't be missed? The answer is to think about a goal in terms of a sequence of
concepts the player has to execute, not a sequence of actions. A concept is something that may be executed by several different sequences of actions. That way, you elevate the puzzle design to a higher level of abstraction.
To provide a descriptive example for concepts, I find it handy to reference my own game Trap Them. You don't have to look it up, just imagine the following. Imagine a system similar to Boulder Dash on the first glance, but every destructible block is affected by gravity and pretty "sticky". It means that a cut out shape falls down and reconnects to the ground again. Now an immediate implication is that this system allows you to drill out something that can resemble a "bottle", or a room with a door that can be closed, to put it in other words. And that is a concept right there! Now you can imagine the following goal. It is a sequence of the 3 following concepts:
Construct a bottle -> Lure the enemies inside and "close" it to keep them inside. -> "Shrink" the bottle until all enemies are crushed.
Now the player who truly understands this puzzle will not remember the goal in terms of the performed actions, but in terms of applied concepts. And he will be satisfied because he realized the idea behind it. And the cool thing is that while the concepts have to be executed in order, there can be a lot of freedom and creativity involved in executing a single concept itself. It means you put 10 different players in front of the screen, and you will get 10 different solutions satisfying the same goal.
Now you can go further and realize that concepts may be
nested since it is a consequence of the concept itself. It means that a concept may be composed of several underlying concepts. Even better, the underlying concepts are not necessarily unique but a variety of them may execute the same higher concept. For example, if the higher concept is to build a bridge, you can imagine that there can be many possible ideas (underlying concepts) how to do that.
The information so far now completes a powerful architectural view of puzzles in general: In short, a puzzle is composed of a
sequence of (nested) concepts. Coming up with concepts (and enforcing them!) is the actual hard part. But in the end concepts are abstractions of everyday life objects and activities. So inspirations are everywhere, you just have to look out for them. As a rough measure to estimate the depth of the mechanics, you can take the amount of concepts they are able to express. So once you have found mechanics which can express a rich set of desired concepts, you know you are on the right track.
I want to conclude that rigid puzzles are not always bad. When you want to teach the player specific actions, it makes sense to make it sure. Also, there are possibly concepts which actually boil down to a single sequence of actions, but you think they are so cool that you want to make them a goal. Here you have to balance your ego with the expectations of the players. If you exaggerate and throw too many of them in, the player may get the impression that there is only one way to play the game, the one and only rigid way the designer intended it to be. But if you left out every rigid concept, you may not show all the cool stuff the game has to offer. Also, you can go nuts with nesting of concepts and breaking players heads. But some may argue that there is more beauty in a goal that is a sequence of pure, simple concept/s, instead of the more complex nested structure. In the end it is a matter of preference what the player finds interesting or enjoys doing/discovering. But that all is relevant when you get to the production ready stages of puzzle design.
Hope it helps.