Modern precision radiotherapy allows small safety margins and dose escalation. Therefore, biological factors become much more important such as CTV delineation and thresholds for organ at risk tolerance. Our aim is to develop image based data mining for exploring voxel-based dose–response relationships in very large patient cohorts. Large numbers of planning CTs are deformably registered to a reference CT. Registration uncertainties are quantified using organ-at-risk contours, dose distributions are smoothed according to these uncertainties and mapped onto the reference. Next outcome measures are correlated voxel-by-voxel with the dose distributions. The resulting correlation maps are tested for significance using a test statistic, e.g. maximum t-value, using randomization to test for significance. We have applied this methodology in several tumour sites and a great strength of this technique is that it allows discovery of sensitive sub-structures of organs. For example, in lung cancer we demonstrated a relationship of dose to the base of the heart with early mortality (1100 patients); while in head and neck cancer, masseter dose correlated most with post treatment trismus. In prostate cancer, obturator dose relates to PSA control. To understand the results, it is important to study inherent correlations in voxel-wise dose distributions that are related to planning techniques that are often ignored in dose-volume based analyses. We conclude that voxel based dose response relationships can be discovered efficiently using deformable registration and novel statistical techniques and that these complement traditional dose–volume analyses, and are suitable for very large patient cohorts.