Implementing your own threadpool will probably mean more work than it's worth. To be honest, from what I've seen of the thread pool in .Net, it's actually a pretty damn good implementation. I'm not sure you could meaningfully implement anything better in a reasonable timespan.
Tasks aren't really necessarily only meant for things that may take a few seconds. Really, a task is just an abstraction. A task is something you want done as an asynchronous operation, whereas a thread is a way to perform the task.
If the TPL actually needs a thread, it's still going to use the thread pool, so you won't have to worry about the overhead of constantly creating/destroying threads. I don't think you should run into any problems with the task-based approach