We present cosmological constraints from the Dark Energy Survey (DES) using a combined analysis of angular clustering of red galaxies and their cross-correlation with weak gravitational lensing of background galaxies. We use a 139 deg2 contiguous patch of DES data from the Science Verification (SV) period of observations. Using large-scale measurements, we constrain the matter density of the Universe as Ω m = 0.31 ± 0.09 and the clustering amplitude of the matter power spectrum as σ8 = 0.74 ± 0.13 after marginalizing over seven nuisance parameters and three additional cosmological parameters. This translates into S8 = σ8(Ωm/0.3)0.16 = 0.74 ± 0.12 for our fiducial lens redshift bin at 0.35 < z < 0.5, while S8 = 0.78 ± 0.09 using two bins over the range 0.2 < z < 0.5. We study the robustness of the results under changes in the data vectors, modelling and systematics treatment, including photometric redshift and shear calibration uncertainties, and find consistency in the derived cosmological parameters. We show that our results are consistent with previous cosmological analyses from DES and other data sets and conclude with a joint analysis of DES angular clustering and galaxy-galaxy lensing with Planck Cosmic Microwave Background data, baryon accoustic oscillations and Supernova Type Ia measurements.