How about applying the shield percentage only to the portion of damage that is actually consumed by the shield?
In you first example, this would mean the following:
- 20 damage will deplete all ten shield points, so they'll be zero after the attack
- Since the shield was dealt 10 damage, the 10% reduction amounts to 1 and leaves 19 damage.
- Next, the ten damage points which were dealt to the shield will be subtracted, so that's 9 damage.
- After reducing the flat armor, you'll end up with 6 hitpoint damage.
In your second example, it's like this:
- 100 damage deplete the one shield point
- Since the shield was dealt 1 damage, the 90% reduction amounts to zero or one, leaving a minimum of 99 damage.
- Next, the ten damage points which were dealt to the shield will be subtracted, so that's 98 damage.
- After reducing the flat armor, you'll end up with 95 hitpoint damage.
Alternatively, you could apply the shield percentage modified by how much percent of the shield are currently available. Let's say, you have 1 / 100 sp, then your 90% damage reduction will actually be 0.9% damage reduction. Or, if you have 50 / 100 sp, that would be 45% damage reduction. Actually, this solution is probably better than the one above.
Also, I wouldn't reduce damage by the amount of shield points lost, because you already
have flat armor and the shield already
does a percentage reduction, so that just sounds like more complexity without a real gain..?