Maintaining axons for an organism's lifetime requires a cytoskeleton that is robust, but also flexible to adapt to mechanical challenges and to support plastic changes of axon morphology. Furthermore, cytoskeletal organization has to adapt to axons of dramatically different dimensions, and to their compartment-specific requirements in the axon initial segment, in the axon shaft, at synapses or in growth cones. To understand how the cytoskeleton caters for these different demands, this review summarizes five decades of electron microscopic studies. It focuses on the organization of microtubules and neurofilaments in axon shafts in both vertebrate and invertebrate neurons, as well as the axon initial segments of vertebrate motor and interneurons. Findings from these ultrastructural studies are being interpreted here on the basis of our contemporary molecular understanding. They strongly suggest that axon architecture in animals as diverse as arthropods and vertebrates, are all dependent on loosely cross-linked bundles of microtubules running all along axons, with only minor roles played by neurofilaments.