Ok, let's start from a premise: If you roll the dice, you'll succeed.
The idea behind this is pretty simple - if your character is one who is capable of doing something, then that's reflected in their skills. Fighter-guy does not get to roll to pick locks and hope he rolls a 20, and mage guy does not get to try to break down a door and hope the same[1]. So this makes the foundational decision into one of whether or not the player gets to roll.
So, why might you not be able to roll?
First and foremost, you might not have the appropriate skill. Simple enough.
Second, you might not have enough skill. Even before we get into the details of what a roll means, there may be some rough tiering of skill levels that says "basic lockpicking is not enough to crack open Loki's security system - you need Epic lockpicking." This could be its own topic, but for the moment, just file away that the possibility exists.
Third, you may be missing a key element. It might be a physical limitation, like trying to hack a computer you can't physically access, or it might be some piece of information, like the language you need to speak.
Fourth, because it might be too trivial to merit a roll. Sometimes success just happens.
Now, given all that, what does a skill roll mean? Now that success is not at question, it's now all about all the other things we talk about that make skill rolls interesting. These are potentially different for any roll, but a few of the big categories include:
Time - Can this be done quickly or slowly? (QUICKLY/SLOWLY)
Quality - Will this be a well-crafted job, or held together with duct tape and spit? (WELL/POORLY)
Style - How good do you look doing it? (STYLISH/MESSY)[2]
Durability - Is this built to last, or is it just barely going to hold together? (DURABLE/FRAGILE)
Consequences - Situationally, what else might go wrong (or right)?[3]
All of these gain some meaning when you have a clear expectation of how things are going to go - all else being equal, that lock is going to take a few minutes to pick, for example.[4] And they gain extra meaning from the context - a few minutes may be too long if the guards are on regular patrol.
Now, if we were going pure diceless, then we could view this as a currency swap. Imagine each of those categories as a switch that could be -, 0 or +. 0 means it's as expected, + means it does well in that category, - means its less good. In this case, the the problem is that the lock needs to be picked QUICKLY, so the player offers a tradeoff, that it will be MESSY (that is to say, it will be obvious upon inspection that the lock has been picked). That brings things to a net 0 (+1 for QUICKLY, -1 for AWKWARD) which is what the character needs to succeed.
When you add dice into the equation, then the dice become the currency - you need to roll well enough to improve on your base success.
Ok, this is getting abstract, so let me ground this: Let's take a basic success-counting system like the Storytelling System. Core mechanic is simple: Roll a bunch of d10s, if any of them shows a success (7+), the roll is a success, with the number of successes coloring the outcome. We're cheating a bit because this system does have the option of failure, but we can handle that.
In this case, we assume that one success is effectively a "0 point" success, goign exactly as expected. Extra successes can be "cashed in" to improve the quality and nature of the roll - so, if you roll 3 successes, you might opt to succeed QUICKLY and STYLISHLY.
This gives players a chance to be awesome a lot, but it also introduces an interesting tool into the GM's arsenal, since the same thinking can be applied to difficulties. That is, the GM can set a few things at their negative values at the outset - a very fragile lock, for example, is going to be MESSY work, and if the player gets only one success, it's going to be MESSY. A second success will be needed to cancel out that (but, importantly, there's no obligation to do so - if the player is ok with it being MESSY, he might use his successes to be QUICK).[5]
Obviously, this leaves a lot of territory uncovered (conflicts and contests jump to mind, though I really like the prospect of contests where opponents push on different axes depending on their priorities) but hopefully the idea is reasonably clear.
1 - Is that a skill? Sure, the same way running or climbing is. There are a range of tasks that don't need the skill, but the skill represents that it's your thing. You break down doors (or run or talk or whatever)
2 - This may be the most mechanically toothless category, but humans are vain, and we like looking like we know what we're doing.
3 - Though, really, EVERYTHING is just a flavor of consequence
4 - Yes, the setting of expectations is powerful mojo, and we'll be getting to some of the mechanical hooks into that.
5 - Did I just suggest that something might be double or triple QUICK? Why yes, yes I did, though what that means is a whole other topic.