Electronic – the difference between a 1st and 2nd Order Digital Filter

What is the physical meaning of "first" and "second order"? ... How do I know if a system is first or second order?

A 1st order system has one energy storage element and requires just one initial condition to specify the unique solution to the governing differential equation. RC and RL circuits are 1st order systems since each has one energy storage element, a capacitor and inductor respectively.

A 2nd order system has two energy storage elements and requires two initial conditions to specify the unique solution. An RLC circuit is a 2nd order system since it contains a capacitor and an inductor

Where do equations (1) and (4) come from?

Consider the homogeneous case for the 1st order equation:

$$\tau \frac{dy}{dt} + y = 0$$

As is well known, the solution is of the form

$$y_c(t) = y_c(0) \cdot e^{-\frac{t}{\tau}}$$

which gives physical significance to the parameter \$\tau\$ - it is the time constant associated with the system. The larger the time constant \$\tau\$, the longer transients take to decay.

For the 2nd order system, the homogeneous equation is

$$\tau^2\frac{d^2y}{dt^2} + 2\tau \zeta \frac{dy}{dt} + y = 0$$

Assuming the solutions are of the form \$e^{st}\$, the associated characteristic equation is thus

$$\tau^2s^2 + 2\tau\zeta s + 1 = 0 $$

which has two solutions

$$s = \frac{-\zeta \pm\sqrt{\zeta^2 -1}}{\tau}$$

which gives physical meaning to the damping constant \$\zeta\$ associated with the system.

The transient solutions are, when \$\zeta > 1\$ (overdamped), of the form

$$y_c(t) = Ae^{\frac{-\zeta +\sqrt{\zeta^2 -1}}{\tau}t} + Be^{\frac{-\zeta -\sqrt{\zeta^2 -1}}{\tau}t} $$

when \$\zeta = 1\$ (critically damped), the solutions are of the form

$$y_c(t) = \left(A + Bt\right)e^{-\frac{\zeta}{\tau}t} $$

and when \$\zeta < 1\$ (underdamped), the solutions are of the form

$$y_c(t) = e^{-\frac{\zeta}{\tau}t}\left(A\cos \left(t\sqrt{1 - \zeta^2}\right) + B\sin \left(t\sqrt{1 - \zeta^2}\right) \right)$$

When given a first order system, why is sometimes equation (2) given, and sometimes equation (3) as the transfer function for this system?

Different disciplines have different conventions and standard forms. Equation (2) looks to me like control theory standard while equation (3) looks like signal processing standard.

Standard forms evolve to fit the needs of a discipline. Further, if a particularly influential person or group develops and uses a particular convention, that convention often becomes the standard. It might be educational to peruse older textbooks and journals to get a sense of how notation and standard forms evolve.

The rate of decay of the impulse response only depends on the distance of the poles of the transfer function to the imaginary axis in the \$s\$-plane, i.e. on the real part of the (generally complex) poles. Remember that for a causal and stable system all poles of the transfer function must be in the left half plane (i.e. the real parts of the poles must be negative), such that the impulse response decays. However, the closer a pole to the imaginary axis the slower does its contribution to the total impulse response decay. The contribution of a complex pole \$s_{\infty}=\sigma+j\omega\$ to the impulse response has the form

$$e^{s_{\infty}t}=e^{\sigma t}e^{j\omega t} $$

which shows that the decay depends on \$\sigma\$, the real part of \$s_{\infty}\$.

This means that in principal even a first order system can have an arbitrarily slow decay of its impulse response. However, for optimal frequency selective filters (Elliptic, Butterworth, etc.) it is the case that higher order filters have poles that are closer to the imaginary axis of the complex \$s\$-plane than low order filters. So the slow decay which you observed for higher order filters is only indirectly related to the filter order. The real reason is the position of the poles close to the \$j\omega\$-axis. These poles result in steeper frequency responses, and sharp transitions in the frequency domain correspond to long (i.e. slowly decaying) impulse responses.