UPS and Inverter power supply sync question

I ended up trying a true sine inverter, and that seemed to do the trick. I guess the ups did not like the square wave from the inverter.

You are describing a Solar Hybrid SOMETHING that may or may not exist in todays vast Marketplace? Step back and revisit a Block Diagram of a Complete Solar Panel Whole House System. You describe several Blocks or Components of the complete system and seem to want all of them in one Device which may be possible today but not practical for the DIY person that dwells within you. It is my impression your main question is of a "Charge Controller" who's function is to charge your Battery using your Solar Panels as the Charging Power Source. Todays Charge Controllers decide for you when the Mains need to take over, when the Battery is fully charged by the Solar Panels, and also when to divert the Solar Panels output to a Dump Load if and when your Batteries are completely charged. The Inverter is the expensive item who's sole purpose in life is to Convert the DC Battery Voltage to a usable AC Voltage ( not necessary for LIGHTING Circuits ONLY ) and Current your household appliances will agree with exactly like your Mains Power from your local Utility Company 230 VAC at 50 Hz I suspect. In an effort to thoroughly answer your question in the form of a recommendation.... Do not look for a single Component that "does it all", if it fails you will loose everything. Rather purchase a quality Charge Controller, a quality Inverter of size and capability to handle your entire anticipated Load, which will indeed be expensive! Most of all, read up / study the topic and fully understand what each individual Block or Component of the Basic,Entire System is responsible for and mostly how they interact to form a Complete System before making any investment. Read, understand, and then make an informed decision / plan to purchase equipment.