The constitutive relationship of concrete under passive confinement of fibre reinforced polymer (FRP) is one of the fundamental requirements for mechanical analysis of FRP-confined concrete members. The existing tri-axial constitutive models are commonly established based on actively confined concrete or uniform passively confined concrete. True tri-axial experiments and corresponding constitutive models for non-uniform passively confined concrete are rare. In this study, based on the pioneering work of true tri-axial testing on concrete cubes associated with passive confinement and the corresponding database assembled by the corresponding author, a plastic-damage based constitutive model for non-uniform passively confined concrete is established. The main parameters for the plastic-damage model i.e., damage variable, hardening/softening rule (cohesion stress) and flow rule (dilation angle) are determined. Non-uniformity factor and maximum lateral confinement stiffness, the key parameters for passively confined concrete, are used to define the cohesion stress and dilation angle curves. The proposed model is validated and then applied in the analysis of FRP fully and partly confined circular columns (uniform passive confinement) and FRP confined square columns (non-uniform passive confinement). The well-matched performance demonstrates the applicability of the proposed model in simulating mechanical response of FRP confined concrete with complex stress fields.