The two numbers you get per frequency are the real and imaginary part of that frequency component in the original signal. These two together are sometimes called a phaser.
You can visualize this by plotting them on a graph where the real component is the X axis and the imaginary component the Y axis. You can add the two components and draw the result as a vector starting at the origin. Now imagine that vector spinning counter clockwise pivoting at the origin at the particular frequency of the sample. The projection onto the real axis would be a sine function over time, which is the contribution of that frequency in your original sample. The particular position your algorithm gave for the spinning vector was just where the signal was at time = 0.
From this you can hopefully see that what you want is the magnitude of that spinning vector. To get that, you square the two components, add them, then take the square root of the result, just like PaulR said. His answer is right, but I was trying to give you some feel for where it came from.
Your real confusion seems to be a fundamental misunderstanding of what DSPs do. DSPs are optimized to perform convolutions. Since a coeficient has to be stored and a multiply-accumulate performed for each point of the convolution, the number of points is limited by memory and available processor time. The convolutions therefore by necessity must be some finite width, so these types of filters are often referred to as finite impulse response, or FIR.
Other than the restriction on the width of the convolution, nothing in the DSP hardware says what you can do with that convolution, or more specifically, what coeficients you can use. All the coeficients together form the function you are convolving a input signal with. They are sometimes collectively called the filter kernel.
There are many possible uses for this basic capability provided by DSPs. Sometimes the desire is to eliminate all content past some frequency while not altering content below that frequency, but that is only one of many useful things a wide digital convolution can do.
However, even when a DSP is used in this way, it is not done with a "window of rectangular shape". There will always be a window of some finite size (that's the basis of a FIR filter), but the shape of that window is rarely rectangular. Using DSP hardware to implement a rectangular filter is rather a waste. Since all coeficients are equal, you can implement this specific case of convolution with a circular buffer, two multiplies, and two adds per sample, regardless of how wide the buffer is. This is sometimes called a "moving average" filter, or "box" filter. For most purposes these don't have very good characteristics. They seem to be used a lot for two reasons: They are the knee jerk reaction of those that didn't pay attention in signal processing class, and they are conceptually easy to implement.
The specific case of a sharp cutoff low pass filter requires the filter kernel to be a sinc function. A sinc in the time domain maps to a rectangle in the frequency domain, and vice versa.
You also seem to be confused in that a FFT is somehow envolved. A fourier transform or lots of other analisys tools may be used to determine what the filter kernel should be, but once the kernel coeficients are determined it's all just a convolution at run time. If you start out knowing what you want to do to a signal in terms of a frequency domain multiplication, then it takes a fourier transform to find the filter kernel that will realize that operation in the time domain as a convolution. However, there are many possible criteria for manipulating a signal, and not all of those may be expressed in the frequency domain. Some may come at you directly in the time domain, in which case no fourier analisys may be needed to determine the filter kernel.
Best Answer
Without math you can't go so far, though....
If you have a periodic signal, you can try to localize the main harmonics (sinewaves) composing the signal; each periodic signal in the real world can be approximated by a sum of sinewaves. And even in this case you are computing a very elementary form of Fourier series decomposition.
Without some math, you can't extract much meaningful information from a time-limited signal.
There is a little exception, though: you can let a spectrum analyzer or the math function of the scope to do the work for you. It can still be useful for several tasks, but if you don't understand the mechanisms, you could get stuck in front of something magic that you don't know how to read.
In any case,
frequency based analysisany application of electronics, especially when involving signals analysis, without a minimum of math gives very poor results.