We study the chemical abundances of the interstellar medium surrounding high-z gamma-ray bursts (GRBs) through analysis of the damped Lyα systems (DLAs) identified in afterglow spectra. These GRB DLAs are characterized by large H I column densities N$_HI$ and metallicities [M/H] spanning 1/100 to nearly solar, with median [M/H]>-1 dex. The majority of GRB DLAs have [M/H] values exceeding the cosmic mean metallicity of atomic gas at z>2 i.e., if anything, the GRB DLAs are biased to larger metallicity. We also observe (1) large [Zn/Fe] values (>+0.6 dex) and subsolar Ti/Fe ratios, which imply substantial differential depletion; (2) large α/Fe ratios, suggesting nucleosynthetic enrichment by massive stars; and (3) low C$^0$/C$^+$ ratios (<10$^-4$). Quantitatively, the observed depletion levels and C$^0$/C$^+$ ratios of the gas are not characteristic of cold, dense H I clouds in the Galactic interstellar medium (ISM). We argue that the GRB DLA represents the ISM near the GRB but not gas directly local to the GRB (e.g., its molecular cloud or circumstellar material). We compare these observations with DLAs intervening in background quasars (QSO DLAs). The GRB DLAs exhibit larger N$_HI$ values, higher α/Fe and Zn/Fe ratios, and higher metallicity than the QSO DLAs. Although these differences are statistically significant, the offsets are relatively modest (N$_HI$ excepted). We argue that the differences result primarily from galactocentric radius- dependent differences in the ISM: GRB DLAs preferentially probe denser, more depleted, higher metallicity gas located in the inner few kiloparsecs, whereas QSO DLAs are more likely to intersect the less dense, less enriched, outer regions of the galaxy. Finally, we investigate whether dust obscuration may exclude GRB DLA sight lines from QSO DLA samples; we find that the majority of GRB DLAs would be recovered, which implies little observational bias against large N$_HI$ systems.