We propose a method of registering 3D images in which many regions have been segmented and labelled. Images in which some regions have been labelled can be registered by generating a vector valued image with a number of planes, one for each individual label class, and applying registration algorithms to the multi-plane images . However, when there are many labels such an approach can lead to unpractically large images. We demonstrate that good results can be obtained by mapping each label value to a vector in a low dimensional space and applying a multi-plane registration algorithm to the resulting vector image. For the approach to work well, the vectors used for each label should be well separated, and chosen in such a way that there is minimal confusion between them. We demonstrate the method by using it to construct statistical shape models by applying a groupwise alignment method to a set of richly labelled 3D brain images. © 2006 IEEE.