The major objective of this work is to develop an efficient finite element analysis (FEA) procedure to simulate wave propagation in fluid-filled pipes accurately. The development of such a simulation technique is essential in the study of wave propagation in pipe networks such as oil and gas pipelines and urban water distribution networks. While numerical analysis using FEA seems superficially straight forward, this paper demonstrates that the element type and refinement used for acoustic FEA have a significant effect on the accuracy of the result achieved and the efficiency of the computation. In particular, it is shown that the well-known, better overall performance achieved with 3D solid hexahedral elements in comparison with 2D-type elements in most stress and thermal applications does not occur with acoustic analysis. In this paper, FEA models were developed taking into account the influence of element type and sizes using 2D-like and 3D element formulations, as well as linear and quadratic nodal interpolations. Different mesh sizes, ranging from large to very small acoustic wavelengths, were considered. The simulation scheme was verified using the Time of Flight approach to derive the predicted acoustic wave velocity which was compared with the true acoustic wave velocity, based on the input bulk modulus and density of air. For finite element sizes of the same order as acoustic wavelengths which correspond to acoustic frequencies between 1 kHz and 1 MHz, the errors associated with the predictions based on the 3D solid hexahedral acoustic elements were mostly greater than 15%. However, for the same element sizes, the errors associated with the predictions based on the 2D-like axisymmetric solid acoustic elements were mostly less than 2%. This indicates that the 2D-like axisymmetric solid acoustic elements are much more efficient than the 3D hexahedral acoustic elements in predicting acoustic wave propagation in pipes, as they give higher accuracies and are less computationally intensive. In most stress and thermal FEA, the 3D solid hexahedral elements are much more efficient than 2D-type elements. However, for acoustic FEA, the results show that 2D-like axisymmetric elements are much more efficient than 3D solid hexahedral elements.