In the time domain

Time domain model of BPSK Costas loop

In the simplest case ${\displaystyle m^{2}(t)=1}$. Therefore, ${\displaystyle m^{2}(t)=1}$ does not affect the input of the noise-reduction filter. The carrier and voltage-controlled oscillator (VCO) signals are periodic oscillations ${\displaystyle f_{ref,vco}(\theta _{ref,vco}(t))}$ with high-frequencies ${\displaystyle {\dot {\theta }}_{ref,vco}(t)}$. The block ${\displaystyle \bigotimes }$ is an analog multiplier.

A linear filter can be described mathematically by a system of linear differential equations:

${\displaystyle {\begin{array}{ll}{\dot {x}}=Ax+bu_{d}(t),&u_{LF}=c^{*}x.\end{array}}}$

where ${\displaystyle A}$ is a constant matrix, ${\displaystyle x(t)}$ is a state vector of the filter, ${\displaystyle b}$ and ${\displaystyle c}$ are constant vectors.

The model of a VCO is usually assumed to be linear:

${\displaystyle {\begin{array}{ll}{\dot {\theta }}_{vco}(t)=\omega _{vco}^{free}+K_{vco}u_{LF}(t),&t\in [0,T],\end{array}}}$

where ${\displaystyle \omega _{vco}^{free}}$ is the free-running frequency of the VCO and ${\displaystyle K_{vco}}$ is the VCO gain factor. Similarly, it is possible to consider various nonlinear models of VCO.

Suppose that the frequency of the master generator is constant ${\displaystyle {\dot {\theta }}_{ref}(t)\equiv \omega _{ref}.}$ Equation of VCO and equation of filter yield

${\displaystyle {\begin{array}{ll}{\dot {x}}=Ax+bf_{ref}(\theta _{ref}(t))f_{vco}(\theta _{vco}(t)),&{\dot {\theta }}_{vco}=\omega _{vco}^{free}+K_{vco}c^{*}x.\end{array}}}$

The system is non-autonomous and rather tricky for investigation.

In the phase-frequency domain

Equivalent phase-frequency domain model of Costas loop

VCO input for phase-frequency domain model of Costas loop

In the simplest case, when

${\displaystyle {\begin{aligned}f_{ref}{\big (}\theta _{ref}(t){\big )}=\cos {\big (}\omega _{ref}t{\big )},\ f_{vco}{\big (}\theta _{vco}(t){\big )}&=\sin {\big (}\theta _{vco}(t){\big )}\\f_{ref}{\big (}\theta _{ref}(t){\big )}^{2}f_{vco}\left(\theta _{vco}(t)\right)f_{vco}\left(\theta _{vco}(t)-{\frac {\pi }{2}}\right)&=-{\frac {1}{8}}{\Big (}2\sin(2\theta _{vco}(t))+\sin(2\theta _{vco}(t)-2\omega _{ref}t)+\sin(2\theta _{vco}(t)+2\omega _{ref}t){\Big )}\end{aligned}}}$

The standard engineering assumption is that the filter removes the upper sideband frequency from the input but leaves the lower sideband without change. Thus it is assumed that the VCO input is ${\displaystyle \varphi (\theta _{ref}(t)-\theta _{vco}(t))={\frac {1}{8}}\sin(2\omega _{ref}t-2\theta _{vco}(t)).}$ This makes a Costas loop equivalent to a phase-locked loop with phase detector characteristic ${\displaystyle \varphi (\theta )}$ corresponding to the particular waveforms ${\displaystyle f_{ref}(\theta )}$ and ${\displaystyle f_{vco}(\theta )}$ of the input and VCO signals. It can be proved that filter outputs in the time and phase-frequency domains are almost equal.^[5][6][7]

Thus it is possible^[8] to study the simpler autonomous system of differential equations

${\displaystyle {\begin{aligned}{\dot {x}}&=Ax+b\varphi (\Delta \theta ),\\\Delta {\dot {\theta }}&=\omega _{vco}^{free}-\omega _{ref}+K_{vco}c^{*}x,\\\Delta \theta &=\theta _{vco}-\theta _{ref}.\end{aligned}}}$.

The Krylov–Bogoliubov averaging method allows one to prove that solutions of non-autonomous and autonomous equations are close under some assumptions. Thus, the Costas loop block diagram in the time domain can be asymptotically changed to the block diagram on the level of phase-frequency relations.

The transition to the analysis of an autonomous dynamical model of the Costas loop (in place of the non-autonomous one) allows one to overcome the difficulties related to modeling the Costas loop in the time domain, where one has to simultaneously observe a very fast time scale of the input signals and slow time scale of signal's phase. This idea makes it possible^[9] to calculate core performance characteristics - hold-in, pull-in, and lock-in ranges.