It is investigated how the nuclear degrees of freedom of the tunneling system (TS) inherent in an amorphous solid influence its dielectric properties. The nuclear quadrupole electric moment of the TS interacting with the crystal field determines the tunneling TS features. At sufficiently high temperatures, the TS is described by the standard tunneling model (STM). At sufficiently low temperatures, this interaction breaks down the coherence of the tunneling. This effect is responsible for the anomalous low-temperature behavior of the sound velocity and low-frequency dielectric permittivity. In addition, the effect explains the anomalous high sensitivity of the dielectric permittivity to the external magnetic field. If the magnetic field is strong enough, the tunneling becomes coherent and the STM can be applied to describe the TS. The approach developed explains the temperature and magnetic field dependence of the real part in the dielectric permittivity, revealed recently in an experiment. We predict an anomalous temperature and magnetic field dependence of the ultrasonic adsorption and thermal conductivity, and in particular, ultrasound absorption and thermal conductivity of glasses below 10 mK.