Fingers off the Pan Knob
The standard panning knob (or balance on a stereo signal) works in a way that it simply changes the levels of the right and left channel according to where you set it. Therefore, on stereo signals it only seems to change the direction of the sound while effectively the “middle point” of the sound doesn’t change. Another effect that it has on stereo signals is to narrow down the stereo width.
With most sample libraries being in stereo and usually quite a bit of room signal on them, panning them will be problematic as it will audibly decrease its stereo width which will reduce its realism on orchestral tracks. Fortunately, most modern sample libraries are recorded with several microphone settings and usually properly seated in the room. In order to change the position, you can pan the close mic (mono signal) to the desired place (within reason) as it is done with live recordings as well but keep the main and surround signals untouched. By that you can gain control over the direction of the sound.
If you need to position a stereo signal without having access to the different mics, you should be using stereo imaging plugins/functions that are caplable of positioning a stereo signal without cutting down its stereo width.