Calcium cobaltite (Ca3Co4O9) is a promising p-type thermoelectric oxide material. Here, we present an approach to optimize the thermoelectric performance of Ca3Co4O9 by controlling the chemical composition and fabrication process. Ca3–xBixCo3.92O9+δ (0.1 ≤ x ≤ 0.3) and Ca2.7Bi0.3CoyO9+δ (3.92 ≤ y ≤ 4.0) ceramics were prepared by Spark Plasma Sintering (SPS). Stoichiometric mixtures of raw materials were combined and calcined at 1203 K for 12 h, followed by SPS at 1023 K for 5 min at 50 MPa. The samples were subsequently annealed at 1023 or 1203 K for 12 h in air. XRD and HRTEM analyses confirmed the formation of the cobaltite misfit phase with minor amounts of secondary phases; SEM-EDS showed the presence of Bi-rich and Co-rich secondary phases. After annealing at 1203 K, the secondary phases were significantly reduced. By controlling the cobalt deficiency and level of bismuth substitution, the electrical conductivity was enhanced without degrading Seebeck coefficients, promoting a high power factor of 0.34 mW m–1 K–2 at 823 K (parallel to the ab planes, //ab). Due to enhanced phonon scattering, the thermal conductivity was reduced by 20%. As a result, a highly competitive ZT(//ab) of 0.16 was achieved for Ca2.7Bi0.3Co3.92O9+δ ceramics at 823 K.