We examine the abundance of molecular hydrogen (H$_2$) in the spectra of gamma ray burst afterglows (GRBs). In nearby galaxies, H$_2 $ traces the cold neutral medium (CNM) and dense molecular star-forming interstellar gas. Although H$_2$ is detected in at least half of all sight lines toward hot stars in the Magellanic Clouds and in åisebox-0.5ex 25% of damped Lyα systems toward quasars, it is not detected in any of the five GRB environments with a similar range of neutral hydrogen column density and metallicity. We detect no vibrationally excited H$_2$ that would imply that the GRB itself has photodissociated its parent molecular cloud, so such models are ruled out unless the parent cloud was <i̊sebox-0.5ex 4 pc in radius and was fully dissociated prior to the spectroscopic observations, or the star escaped its parent cloud during its main-sequence lifetime. The low molecular fractions for the GRBs are mysterious in light of their large column densities of neutral H and expectations based on local analogs, i.e., 30 Doradus in the LMC. This surprising lack of H$_2$ in GRB damped Lyα absorbers indicates that the destruction processes that suppress molecule formation in the LMC and SMC are more effective in the GRB hosts, most probably due to a combination of low metallicity and an FUV radiation field 10-100 times the Galactic mean field. These inferred conditions place strong constraints on the star-forming regions in these early galaxies.