The effects of impurities and magnetic fields on the quantized conductance G of a narrow constriction are studied. Both the cases when the impurity is inside and outside the effective width of the constriction are considered. We show that a strong attractive scatterer outside the effective width of the constriction can give rise to a dip structure in G. The occurrence of such a dip structure depends on the transverse location of the impurity. This is different from the effect of a weak scatterer inside the constriction which causes a dip structure in G only when the Fermi level is just below a transverse subband bottom. Our finding suggests a possible new scattering mechanism for quantum transport in narrow constriction systems. In addition, we demonstrate that in applying a perpendicular magnetic field, the backscattering of electrons from an impurity is suppressed, leading to better conductance quantization.