Next, we take away the sword. So now we have to make sure that the player can avoid all enemies, instead of requiring them to kill some.
this last point seems like the kind of benefit people are talking about to a regressive system. Rather than the game being harder because your sword doesn't do as much damage to the enemies, the game is harder because you have no sword. You have to develop a whole new strategy for every single enemy encounter, including the foes that used to be super easy level-1 type creatures.
Also, for the other examples earlier in your post, you're assuming some kind of strange linear progression through the game involving new areas and the gradual stripping of every single ability. Not saying you couldn't do that but regression as a mechanic has far broader applications than that - such as repeatedly traversing the same space with diminished abilities, or not taking away every single damn ability a character has, but just a few - which ones and when depends on the game, that's where the whole role of the game designer comes in and shit. Not just algorithmically turning the player character into a paraplegic.
You could also have both progression and regression in a game, perhaps simultaneously to balance one another out, or as a kind of parabola where the player reaches their peak power at a mid-game climax and then begins losing abilities from there on out. And if you want to give them back their powers eventually don't do it before the final boss and make it a cakewalk. Make them fight the final boss crippled as they come, and give them the power-fantasy shit as a reward, using it to blast their way out of the collapsing facility or something.
It just ain't like progression has to be a linear Get Power or a linear Lose Power. Make it a rollercoaster or something.
edit: also none of that would "invert" the learning curve since you'd still have to learn how to deal with each of the situations without the abilities you once relied on. Figuring out how to get through a cracked wall without bombs is still an ability that can be obtained, just not as an item in the game but as a process in the mind of the player. You have to trick a ram enemy into charging into it, or find some other mobile explosive that isn't constantly on your character's person, or or or - these things are part of the learning experience as much as how to use any tool given the player. And, as Tromack said, knowledge isn't something that you can actively 'lose' due to processes of the game. Maybe it can become obsolete, but getting a new, more powerful sword renders your old sword obsolete as well. I see no reason why not to try changing the priority of things which are made obsolete in games - information, items, abilities, etc.