the op-amp is correctly wired up with in an inverting circuit configuration. because of the negative feedback through passive components, the "-" terminal is a virtual ground. the node equations (\$V_2\$ is the voltage at the node where are \$R_1\$, \$R_2\$, \$C_4\$, and \$C_3\$ are connected) are:
$$ \left(\frac{1}{R_1} + \frac{1}{R_2} + sC_4 + sC_3 \right)V_2 - sC_4 V_\text{out} = \frac{1}{R_1} V_\text{in}$$
$$ sC_3 V_2 + \frac{1}{R_5} V_\text{out} = 0 $$
from that, i get
$$ \begin{align}
A(s) \triangleq \frac{V_\text{out}}{V_\text{in}} & = \frac{-\frac{1}{R_1} s C_3}{\frac{1}{R_5}\left(\frac{1}{R_1} + \frac{1}{R_2} + sC_4 + sC_3 \right) + (sC_4)(sC_3) } \\
\\
& = \frac{-\frac{1}{R_1 C_4} s}{\frac{1}{R_5 C_3 C_4} \left( \frac{1}{R_1} + \frac{1}{R_2} \right) + \frac{C_4 + C_3}{R_5 C_3 C_4} s + s^2 } \\
\\
& = \frac{-H \omega_0 s}{s^2 + \frac{\omega_0}{Q} s + \omega_0^2} \\
\end{align} $$
equating the corresponding coefficients...
$$ \omega_0^2 = \frac{1}{R_5 C_3 C_4} \left( \frac{1}{R_1} + \frac{1}{R_2} \right) $$
$$ \frac{\omega_0}{Q} = \frac{C_4 + C_3}{R_5 C_3 C_4} $$
$$ H \omega_0 = \frac{1}{R_1 C_4} $$
i think the intent, in the lecture notes posted in the question is that \$ \omega_0 \triangleq 2 \pi f_\text{m} \$
so let \$ C_3 = C_4 \triangleq C \$ and let \$ k \triangleq \omega_0 C \$.
then $$ \frac{1}{R_1} = H \omega_0 C $$ $$ \frac{1}{R_2} = (2Q - H) \omega_0 C $$ $$ \frac{1}{R_5} = \frac{1}{2Q} \omega_0 C $$.
so plug this in for \$R_1\$, \$R_2\$, and \$R_5\$ and see if equality in the three "corresponding coefficients" equations above is met. if so, the transfer function, as given in the question, is correct.
The LM324, while a brilliant achievement with 1970s transistors, has one well known bug - actually documented in its datasheet. This answer is based on a guess that you are running into this bug.
Some people sneer at it because of limitations like this - but it is still a fine opamp if you design to its limitations.
Its Class B output stage is specifically designed for low power but it is asymmetric : that is, it can pull up (towards V+) quite strongly, but cannot pull down very effectively. This avoids excessive current consumption where both output transistors might be momentarily turned on together (as in the bipolar 555) - a possibility given the relatively low speed of these transistors.
(See Table 6.5, page 6, Output Current section, in the datasheet - at 5V it can source 20mA but only sink 8 uA, so the pullup is 2500 times stronger)
This combination of circumstances : Class B output stage, slow transistors, asymmetric strength, gives it a particularly bad case of crossover distortion on high frequency signals, there is a region where both output transistors are off and the output voltage is effectively undefined.
Run a sinewave at a few kHz through this filter (unit testing is as valid in hardware as it is in software!) and you'll see large bites taken out of the output waveform.
The (documented if you look for it ... EDIT ... it's in section 7.4 on page 11, and my memory of the problem is slightly off) fix is to force the output stage into Class A, with a few kilohms of pulldown resistor to V- (I suggest 1K at this low voltage). Now the pulldown transistor never has to turn on at all (though no harm done if it does) and the much stronger pullup transistor is always in control, pulling against the resistor.
Thus the 324 is designed to be used successfully in Class B for low speed low power designs, or Class A for faster designs.
The downside to Class A is obviously increased power consumption - if that matters, nowadays you can choose a better opamp.
Best Answer
Quick Synopsis
Let's quickly summarize what I think I understand. It appears you have the following specifications:
(I don't have the available voltage rails. But we can ignore that. And we don't need to worry about the opamp, yet.)
First up is to compute a few details in order to see how to arrange the band-pass. There is a serious choice to be made that will be quite different, depending upon the results. This part I think you already know. But it's worth stating, explicitly. So here goes:
From this last item, I conclude that your approach using a combination of a low-pass and a high-pass filter is correct. Had the fractional bandwidth been a lot less, say below 1 or better still below .8, then there may be an argument instead for combining two bandpass sections, instead. But that's not the case. So you are correct to break things up the way you did.
So far, so good.
No Input Spec -- Assuming Signal Generator Only
You didn't specify what's driving the system. No, I don't mean the signal generator you used. I mean the real thing that will be driving it when you actually get to making use of it.
Sometimes, you need to AC couple into the filter. If so and if the first section is the low-pass section, you'll need to provide a DC path to ground for the (+) input. I don't see a DC blocking capacitor in your case. And obviously, you didn't need one for the simulation. But I've no idea if you do in the actual circuit. If what's driving this circuit has a DC bias to it, you may very well require DC blocking. Just FYI.
Also, you didn't specify the allowable loading that the first section presents to the actual source. Nor the source impedance, itself. So this means I'm in the dark about these details.
But let's assume for now that this is only a proof that you can do a design and that you'll be using the \$50\:\Omega\$ output of the signal generator; with the only purpose in all this is to verify design capabilities.
Section Order and Other Assumptions
I'm going to just assume you know what you are doing and like the low-pass filter as the first section and want the high-pass filter as the second section. There are considerations, such as the the noise of typical opamps related to their 1/f and bandpass regions, and so on. But I'll just assume you've already made those evaluations and I'll just move on.
I'm also going to assume you are looking for a maximally flat (Butterworth) design. This is important because Sallen-Key equal-value component designs have a fixed relationship between voltage gain and damping (\$\zeta\$ or alternately \$Q\$.) You don't get to pick the voltage gain if you are looking for Butterworth. If you choose to pick the gain, then you don't get to choose the damping.
Broadly speaking, you have to select a voltage gain less than 3, under all possible circumstances. There's no alternative to that. Since the \$\sqrt{10}\gt 3\$, you cannot get \$A_v=10\$ with two of these sections. Just can't happen. Adding some gain at the end is okay. But you will need a third section. There's no escaping it.
Low Pass Design
The low-pass filter has a cut-off at \$f_{_\text{C}}=500\:\text{Hz}\$.
Because capacitors aren't usually found in E24 steps, you usually decide on your capacitor values, first. They go in broad steps. In this case, I might select \$47\:\text{nF}\$. The nearby resistor value would then be \$6.8\:\text{k}\Omega\$. With that combination, \$f_{_\text{C}}\approx 498\:\text{Hz}\$. Which is close enough.
The gain of the stage is already set by the required damping to meet a Butterworth filter: \$A_v=3-\sqrt{2}\approx 1.586\$. It's determined. You don't get to select it. Sorry.
(The same will be true of the high-pass filter. So we can already say that you will need a final, third section with \$A_v\approx 4\$ in order to reach your final goal of \$A_v=10\$.)
From this, the low-pass section will be:
simulate this circuit – Schematic created using CircuitLab
(The actual computed value is \$R_4=22.845\:\text{k}\Omega\$. But I rounded it down. The price I paid for that is a slightly lower voltage gain of \$A_v\approx 1.564\$ and a slight change in the damping. Oh, well. Acceptable.)
High Pass Design
The high-pass filter has a cut-off at \$f_{_\text{C}}=20\:\text{Hz}\$.
In this case, I might select selecting \$220\:\text{nF}\$. The nearby resistor value (from E24 series) would then be \$36\:\text{k}\Omega\$. With that combination, \$f_{_\text{C}}\approx 20.1\:\text{Hz}\$. Which is close enough.
As before, the gain of the stage is already set by the required damping to meet a Butterworth filter: \$A_v=3-\sqrt{2}\approx 1.586\$. It's determined.
From this, the high-pass section will be:
simulate this circuit
(Given that both sections had slightly less gain than might have been, ideally, the final, third section will now need \$A_v\approx 4.1\$ in order to reach your final goal of \$A_v=10\$. A slight adjustment.)
Summary
At this point, you only need to add a simple section to get close to \$A_v\approx 4.1\$. A simple approach:
simulate this circuit
So the final Butterworth result is:
simulate this circuit
And here's LTspice's result from the above schematic (using LT1800's for the opamp choice):
Note that the peak is right at \$20\:\text{dB}\$ and where it should be, given your specs.