Same colour, two objects
Every pixel is three numbers. When a jacket and the sky behind it share those numbers, no tolerance can separate them — the answer is to say where, not to measure harder.
The selection only knows values
A colour selection asks one question of each pixel: how close is it to the sampled colour, in hue, chroma and lightness? It has no notion of an object. A blue jacket photographed against a blue sky is, to that question, one blue region with a slightly darker patch in it, and every setting that catches the jacket's shadowed folds also catches the sky.