We present a comprehensive study to measure the locations of gamma-ray bursts (GRBs) relative to their host galaxies. In total, we find the offsets of 20 long-duration GRBs from their apparent host galaxy centers by utilizing ground-based images from Palomar and Keck and space-based images from the Hubble Space Telescope (HST). We discuss in detail how a host galaxy is assigned to an individual GRB and the robustness of the assignment process. The median projected angular (physical) offset is 0.17’’ (1.3 kpc). The median offset normalized by the individual host half-light radii is 0.98, suggesting a strong connection of GRB locations with the UV light of their hosts. This provides strong observational evidence for the connection of GRBs to star formation. We further compare the observed offset distribution with the predicted burst locations of leading stellar-mass progenitor models. In particular, we compare the observed offset distribution with an exponential disk, a model for the location of collapsars and promptly bursting binaries (e.g., helium star- black hole binaries). The statistical comparison shows good agreement, given the simplicity of the model, with the Kolmogorov-Smirnov probability that the observed offsets derive from the model distribution of P$_KS$=0.45. We also compare the observed GRB offsets with the expected offset distribution of delayed merging remnant progenitors (black hole-neutron star and neutron star-neutron star binaries). We find that delayed merging remnant progenitors, insofar as the predicted offset distributions from population synthesis studies are representative, can be ruled out at the 2×10$^-3$ level. This is arguably the strongest observational constraint yet against delayed merging remnants as the progenitors of long- duration GRBs. In the course of this study, we have also discovered the putative host galaxies of GRB 990510 and GRB 990308 in archival HST data. Partially based on observations with the NASA/ERA Hubble Space Telescope, obtained at the Space Telescope Science Institute, which is operated by the Association of Universities for Research in Astronomy, Inc., under NASA contract NAS 5-26555.