In this paper, we look at examples of linguistic change in which the structure associated with the original element can help us understand aspects of both the diachronic process and the synchronic outcome of change. We first consider a range of phenomena that have resisted formal analysis because they show mixed-category behaviour. We argue that these elements have undergone changes that we would expect to lead to complete decategorialisation, but the synchronically odd behaviour arises because structural change that would be expected to take place, given what we know of decategorialisation, has failed to occur. We argue that these cases can usefully be thought of as instances of ‘structural persistence’, and that Plank's (1995) notion of Systemstörung ‘system disturbance’ offers a helpful way of looking at these cases. We then put this type of structural persistence alongside other types of change that have been identified in the literature as involving persistence at a structural level, and argue for a renewed focus on the role of structure in language change, including in instances of change that have a semantic component. We argue that the role played by structure is necessary to understand constraints on change, motivations for change, and synchronic mixed behaviour.