Partition Coefficient
Drop a molecule into a jar holding both oil and water and it does not pick a side. It distributes itself between the two and settles at a ratio, and that ratio is the partition coefficient — usually quoted on a logarithmic scale, where a positive number means the substance prefers oil and a negative one means it prefers water.
The preference matters twice over. Inside the bottle it decides which phase of an emulsion an ingredient is really sitting in, which is not always the phase the front of the box implies. On skin it decides whether the molecule will leave the product at all, because the way through the outermost layer is a lipid one — the mortar packed between the dead surface cells. Something that will not give up water for oil has trouble entering it.
More is not better, though. A molecule so oil-loving that it settles happily into that mortar has no reason to move on into the watery layers underneath. It accumulates and stops. The substances that travel are the ones with divided loyalties.
This is also the lever a formulator pulls, and it works against intuition. Improving the solvent until an active dissolves beautifully can reduce how much of it leaves the vehicle, because the molecule is now perfectly comfortable where it is. Which is the short answer to why “it contains X” and “X gets into skin” are two different sentences.