What matters for FM is an instantaneous quantity: the derivative or increment of the instantaneous phase, i.e. how the phase changes from sample to sample.
What is usually called frequency, and has the same value in boring cases like an unvarying sinusoid, is the inverse of the period of the resulting sound as someone would hear it, which is not instantaneous and needs on the order of one period's worth of samples (usually more) to estimate, for a certain definition of period (e.g. interval between zero crossings) or a certain mathematically reasonable calculation (e.g. estimating the frequency of the strongest component from a fixed-length window of past samples).
What is usually called frequency, and has the same value in boring cases like an unvarying sinusoid, is the inverse of the period of the resulting sound as someone would hear it, which is not instantaneous and needs on the order of one period's worth of samples (usually more) to estimate, for a certain definition of period (e.g. interval between zero crossings) or a certain mathematically reasonable calculation (e.g. estimating the frequency of the strongest component from a fixed-length window of past samples).