Electronic – State Space model for three-loop circuit

You misunderstood state-space representation. While the standard form you've posted indeed targets linear systems but this doesn't mean the representation can't handle nonlinear system. Probably one of powerful tools of this representation is its capability for handling nonlinear system. Let's see how the system is represented in the state space (i.e. no matrix form).

Let's define \$q_1=x, q_2=\dot{x},q_3=y, q_4=\dot{y},q_5=\theta, q_6=\dot{\theta}\$, therefore, we obtain

$$ \begin{align} \dot{q}_1 &= q_2\\ \dot{q}_2 &= F_E\cos(\psi)\sin(q_5) + F_E\sin(\psi)\cos(q_5) +F_s\cos(q_5)\\ \dot{q}_3 &= q_4\\ \dot{q}_4 &= F_E\cos(\psi)\cos(q_5) - F_E\sin(\psi)\sin(q_5) - F_s\sin(q_5) \\ \dot{q}_5 &= q_6\\ \dot{q}_6 &= F_E\sin(\psi)(l_1+l_n) - l_2F_s\\ \end{align} $$ Now the system can be represented generally as

$$ \dot{q} = f(q, u), \quad q \in \mathbb{R}^3, u \in \mathbb{R}^3 $$ which can be solved numerically. I can solve the system if you provide me with the actual parameters of the system.

There are several possible state variable presentations for the same circuit, but I guess you want state variables which are measurable quantities of the circuit. In that case you should use inductor currents and capacitor voltages as the state variables.

Capacitors have stale law dUc/dt = I/C where Uc is the cumulated voltage over the capacitor and I is the charging current. Inductors have dI/dt = U/L where I is the cumulated current through the inductor and U is the voltage over the inductor.

The series RLC circuit:

You can write the wanted equations with these laws. Express the voltage over the inductor with your V(t), I, R and Uc. Divide it by L and there's your expression for dI/dt, the right side of the first state equation.

The second state equation is simply (as already written) dUc/dt = I/C.

Circuit analysis programs which do transient analysis generate the equations automatically and solve them numerically. You must enter the initial values of the state variables i.e. inductor currents and capacitor voltages or let them be zero.