We investigate properties of the interstellar medium (ISM) in galaxies hosting long-duration γ-ray bursts (GRBs) from an analysis of atomic species (Mg$^0$, Fe$^0$) and excited fine-structure levels of ions (e.g., Si$^+$). Our analysis is guided primarily by echelle observations of GRB 050730 and GRB 051111. These sight lines exhibit fine-structure transitions of O$^0$, Si$^+$, and Fe$^+$ gas that have not yet been detected in intervening quasar absorption-line systems. Our results indicate that the gas with large Mg I equivalent width (e.g., GRB 051111) must occur at distances >åisebox-0.5ex 50 pc from GRB afterglows to avoid photoionization. We examine the mechanisms for fine- structure excitation and find that two processes can contribute: (1) indirect UV pumping by the GRB afterglow provided a far-UV intensity in excess of 10$^6$ times the Galactic radiation field; and (2) collisional excitation in gas with electron density n$_e$>10$^4$ cm$^-3$. The observed abundances of excited ions are well explained by UV pumping with the gas at ri̊sebox-0.5ex a few hundred pc from the afterglow for GRB 051111 and r<100 pc for GRB 050730, without invoking extreme gas density and temperature in the ISM. We show that UV pumping alone provides a simple explanation for all reported detections of excited ions in GRB afterglow spectra. The presence of strong fine-structure transitions therefore may offer little constraint for the gas density or temperature. We discuss additional implications of UV pumping including its impact on chemical abundance measurements, new prospects for observing line-strength variability, and future prospects for studying the gas density and temperature. Finally, we list a series of criteria that can distinguish between the mechanisms of UV pumping and collisional excitation.