Sometimes - such as with a ball-point pen or 5H pencil - you'll be struggling to get much variation. A softer pencil or a flexible gold nibbed pen will respond better, giving a strong line as you press, and lifting off for the faintest of marks as you reduce the pressure. With charcoal or a chisel-point pencil, varying the angle of the tip can be used to create great variation in line width.
Context is also important, just as perceived volume is dependent upon background noise: a gray line will look heavier on a bright white than a mid-gray paper, or in the midst of delicate marks compared to strong, energetic marks.

