It is a pretty low parts count design, but it is by no means simple. It helps to understand how the chip works inside to see what pins are more sensitive to stray EMI than others before doing a layout. Then you can determine where coupling to ground is useful and when it can interfere. ( This is just learning curve info not specific to your layout)
Although many students use the physical layout approach, it defies logic and when you want to understand how it works, you need to use logic in your schematics not just physical layout. This is most useful when it doesn't work.

vs

When you discover the load regulation is poor due to the layout of the external transformer in relation to RFB feedback, which schematic will you follow and how will you discover why it fails in your layout. Unfortunately what schematics do not show is the equivalent circuit of a ground plane and its coupling. This is why a more "Logical" schematic is important and rules for small inductive loops is important and coupling to ground planes. Although I concur with your aadvice given, this design will be very noisy and suffer in regulation becuase of how the boundary mode processing works inside the chip. A more reliable design will use large voltage feedback thru a tertiary winding (which they use in msome aqpplications)
It may work, but how well is the question.
EMI radiation, Load regulation, Step response and ripple are important measures of performance which are affected by this clever chip's internal processing of the current waveform using the MOSFET resistance. When it fails you can try to understand how it works. With your schematic, all you can do is see that the parts are connected to the right pin.
These basics although may seem tedius to you, they are critical to getting the perfromance offered by the chip. Layout is critical in this design because of the simplified external feedback of transient switching. I would use a small planar SMD torroidal transformer such as the one used by Linear Technology and follow their layout guidelines.
First of all, before answering your questions, let me make some suggestions about the layout. I have seen your previous post and this has been a good attempt, but:
1.- The "Switching Loog" is too large. (Switching loop is Vin, Trafo, Q1 and R8, you call it,"high current loop", maybe it is no quite precise) And moreover, this loop include the control area. This is really EMI problem!! I suggest something like this.
(Think in current loops always!)
Make smaller the switching loop
- Place R8 near from C1. Really near.
- Rotate Q1 90º clockwise
- Place R6 and C6 as close as possible (acap) from Trafo pins.
After that,
- Think about the "Gate net", "Sense net" and "pin 15 from controller net", It must be route separatly! The "gate net" is a radiant net and the others ones are sensitive nets. Route them.
- Route Vin and Enable controller nets.
- Complete the route of primary
(The controller has not a Decoupling Capacitor ? Are you sure of this?)
2.- A connector for trasformer sounds a problems maker. Could you solder your Custom Transforme in the PCB? It would be better. If you cann't, I you sugggest a 2 row 2 columns conectors. In this way you can gain more space between primary and secondary, and more space between trafo pins.
OK and now. Your questions.
1.- Yes, try to keep separate switching ground from control ground, but connect to the same ground, in your case Bottom layer. For this, try to keep the "switching loop" components together according the placement I suggest.
Connecting R8 to Vin through one track and then one via to GND is not a good idea in your current layout.
2.- Ummm... I would try another layout before consider this question.
3.- I cann't undestand this question. What do you mean?
4.- As a rule, It better If you fill the empty area with hatched copper. It improve the etching process in PCB manufactureing.
5.- As a starting point, fill as much as you can. Do you need consider any electrical isolation between primary and secondary.
Good luck!
Best Answer
No, just putting some hot glue over dangerous points on a PCB is not enough. You will never pass safety certification that way. You need some kind of enclosure such that probing fingers can't get at the hot spots. Saftey standards will usually require a fuse in series with the AC hot line as soon as possible after it enters your box, then you can connect that to a PCB. The rest of the system must mechanically ensure that those nodes are ineccessible.
If this is a one-off for your lab bench, then hot glue, electrical tape, epoxy, etc is OK, but you have to keep in mind how these things can fail and treat everything accordingly. When working with AC power line on a bench, it's a good idea to use a isolation transformer. That adds one level of safety in that there won't be a conduction path from any of the high voltage to ground. However, keep in mind this changes if you connect a grounded scope, computer communication cable, etc, to any part of the circuit.
Something to consider: Don't use a 5V relay, use one instead intended to run directly off the AC line. This may lower the current requirements of the 5V supply so that you don't need a transformer at all anymore. For a few mA you can use a capactive charge pump, which might be enough to run your micro. In that case the whole circuit will be floating over the AC line voltage range, so you have to consider all of it dangerous. But, if this will ultimately be a sealed unit that only activates a relay, then that should be fine. After all, every AC line power circuit will have some parts at line voltage. The issue is how to deal with that properly.
Another method is to get a off the shelf "power brick" to embed into your product. These are switching power supplies that make isolated DC, in your case 5V, that you can then safely run the rest of your circuit on. In that case it would be easier to use a 5V relay as you originally said.
If your power requirements are only a few Watts or less, then you can push the power supply external. In that case you use a wall wart type power supply and only the isolated DC ever enters your unit. These are small sealed units with only the AC prongs exposed and come pre-certified to various safety specs. Customers understandably find them annoying, but the economics usually dictate something like that for up to around 5-10 Watts.