Considering the note under the drawing, I decided to omit R1 and R2
from the circuit
R2 needs to be shorted to get this to work.
Op-amps don't work properly without feedback and that's what the line to pin 16 provides. There's probably a clue in the name of the pin if you look.
EDIT about power rails
I'm mentioning this because I think your AD8055 is powered from +5V and 0V - this will cause the problems listed below (if not using a negative rail) and I don't think the AD8055 is rated to run below a supply rail of +/-4V. Looking at your picture of the output, it appears to be bottoming out at somewhat above +500mV and rather similar to what the implied lowest output level would be on this device in the data sheet.
Just in case your supply regime is wrong, this is how it should look (different chip but same principle): -
Note the negative and positive rails on the op-amp because Iout is positive for a positive reference and for the Iout pin to be held at virtual earth, the op-amp's output must fall below 0V. Here's probably a more explicit example: -
Note that for a Vref voltage of (say) +2.5V, maximum output will be virtually -2.5V with lower values heading to 0V. At no time will the output be positive unless you use a voltage reference that is negative. This is a multiplying DAC configuration and inversion is inherent.
A very nicely presented 1st question (or 100th or ...).
Lots of detail to assimilate but it all seems relevant and useful if a good answer is to be found. I cannot spend the time needed now on this but will throw in a few comments and see what others have said later.
I spent about 15 minutes just going to and fro over the circuits and layouts and doing some basic sanity checking. I'm sure your rule checking would have eliminated basic errors.
I have NOT tried to work out what your fault may be caused by specifically - and suspect that it may be a hard fault or misthought rather than the design areas touched on below. BUT any of the following may relate.
Have you tried placing the whole PCB on a PCB ground plane? Can help heaps with single sided. May not.
The two unrouted nets shown presumably have wire links added by hand. (If not that would be an easy fix :-) )
A single side board MAY be doable but with such a complex beast with two switchers and the ability for feedback between them you'd need real care, a scope glued to your right hand and some luck. Even a double sided board (which is about as cheap and quick from many board houses) costs much the same.
A problem is (which may have led to a problem that you get) that the IC seems to have pinouts which assume you can route across the IC with ease so that critical current loops have little area. Because you are on 1 layer this is not true and you have several such loops that more or less overlap and seem to invite disaster.
The obvious ones to minimise to start are the two inductor loops p7-L1-p15 and p16&p17-L2-p14.The L1 loop involves an added jumper and how you route this may have an effect.
Noise getting into the feedback dividers can be bad news indeed. I see you have used c5 across R4 as per their circuit but have no cap across R8 - shown as Copt on one of their circuits and not on another. Simplistically this passes fast load transients or noise that affects output into the feedback pin at a greater rate and level than you get from the divider. Presence or absence in SOME designs is life or death.
Draw lines on printouts of the layout with different coloured markers as to where the loops seem likely to be that are used by different processes (Inductor currents, feedback dividers, ...). (Draw on a screen if that works for you - I find paper and markers more powerful). You can then see likely interactions and any loops that have large open front doors for noise / cross coupling to rush in and out of.
More later maybe.
Best Answer
Step 1. To get 15V, you want a power supply higher than 12V. Well, the NE555 has an output which can drive about 100 ma. Use this to power a voltage doubler circuit (2 diodes, 2 capacitors) and you'll get a DC supply slightly less than 24V. Searching for "NE555 voltage doubler" should cover the details.
Step 2. Use this supply to power a low-current opamp with a gain of 2 driving the varicaps,