Firewall – Are there other application layer firewalls like Microsoft TMG (ISA) that do advanced http rules

You probably don't need to retain the original IP in the actual TCP header. Many reverse proxy setups will allow passing the original IP address the request was made from as an HTTP header. (See for example: Squid configuration directive forwarded_for.)

Apache with mod_proxy would also do the trick, and will also add X-Forwarded-For headers for reverse-proxied traffic. This might be convenient since you already have Apache running on the machine.

There's nothing to stop your workgroup machines using TMG as their proxy, but you won't be able to authenticate the users using Integrated Authentication in IE or the TMG client.

If you configure the machines as proxy clients (i.e. by configuring the proxy address in the browser) and there are no rules that allow anonymous connections then your workgroup users will receive a logon prompt from their browser - and they will need a domain account. This is likely to irritate users and if you're going to have to give each user a domain account then you might as well join the machines to the domain.

If you want the workgroup machines to all have the same set of restrictions (perhaps a different set of restrictions from your domain joined machines) then you could ensure that they all have IP addresses in a specific range and then restrict traffic based on that range. You would need to allow the connections to be anonymous though.