Swift captured for the first time a smoothly rising X-ray re-brightening of clear non-flaring origin after the steep decay in a long gamma-ray burst (GRB): GRB081028. A rising phase is likely present in all GRBs but is usually hidden by the prompt tail emission and constitutes the first manifestation of what is later to give rise to the shallow decay phase. Contemporaneous optical observations reveal a rapid evolution of the injection frequency of a fast cooling synchrotron spectrum through the optical band, which disfavours the afterglow onset (start of the forward shock emission along our line of sight when the outflow is decelerated) as the origin of the observed re-brightening. We investigate alternative scenarios and find that the observations are consistent with the predictions for a narrow jet viewed off- axis. The high on-axis energy budget implied by this interpretation suggests different physical origins of the prompt and (late) afterglow emission. Strong spectral softening takes place from the prompt to the steep decay phase: we track the evolution of the spectral peak energy from the γ-rays to the X-rays and highlight the problems of the high latitude and adiabatic cooling interpretations. Notably, a softening of both the high and low spectral slopes with time is also observed. We discuss the low on-axis radiative efficiency of GRB081028 comparing its properties against a sample of Swift long GRBs with secure E$_γ,iso$ measurements.