Studies of the cosmic gamma-ray bursts (GRBs) and their host galaxies are now starting to provide interesting or even unique new insights in observational cosmology. Observed GRB host galaxies have a median magnitude Råisebox-0.5ex 25 mag, and show a range of luminosities, morphologies, and star formation rates, with a median redshift zi̊sebox-0.5ex 1. They represent a new way of identifying a population of star-forming galaxies at cosmological redshifts, which is mostly independent of the traditional selection methods. They seem to be broadly similar to the normal field galaxy populations at comparable redshifts and magnitudes, and indicate at most a mild luminosity evolution over the redshift range they probe. Studies of GRB optical afterglows seen in absorption provide a powerful new probe of the ISM in dense, central regions of their host galaxies, which is complementary to the traditional studies using QSO absorption line systems. Some GRB hosts are heavily obscured, and provide a new way to select a population of cosmological sub-mm sources. A census of detected optical tranistents may provide an important new way to constrain the total obscured fraction of star formation over the history of the universe. Finally, detection of GRB afterglows at high redshifts (z>6) may provide a unique way to probe the primordial star formation, massive IMF, early IGM, and chemical enrichment at the end of the cosmic reionization era.