The structure of the phenol-argon cluster (PhOH-Ar) in high-n Rydberg states is investigated by the newly developed technique of mass analyzed threshold ionization detected infrared (MATI-IR) spectroscopy. This method selectively measures IR spectra of molecular clusters in very high-n Rydberg states (n > 100) utilized in zero kinetic energy (ZEKE) photoelectron and MATI spectroscopy, whose ionic cores are essentially the same as the corresponding bare cation. The MATI-IR spectrum exhibits only the free OH stretching vibration (V-OH(pi)) when the pi-bound cluster of the neutral ground electronic state (S-0) is resonantly excited via the S-1 origin to Rydberg states converging to its adiabatic ionization energy level, IE0(pi). When Rydberg states converging to vibrationally excited levels of the local p-bound minimum are prepared, in addition to V-OH(pi) also the hydrogen-bonded OH stretching vibration (V-OH(H)) of the H-bonded global minimum is observed in the MATI-IR spectra, even for vibrational excitation of only 14 cm(-1) above IE0(pi). These results show that the pi -> H site switching reaction of the Ar ligand from the aromatic ring to the OH group proceeds only from vibrationally excited states in the p-bound cation core with a small barrier of less than 14 cm(-1) from IE0(pi). On the other hand, directly photoionized PhOH+-Ar shows both V-OH(H) and V-OH(pi) in the IR spectra, even when it is just ionized to IE0(pi). This result implies that the ionization-induced pi -> H site switching occurs without excess energy in the H-bound or pi-bound cations, in contrast to very high-n Rydberg states converging to levels of the pi-bound cation. The different efficiencies of the site switching for the Rydberg ion core and the bare ion and the mechanism for the pi -> H site switching are interpreted by direct ionization from the pi-bound to the H-bound structures in addition to the conventional vertical ionization and transitions to high-n Rydberg states.