This type of solution exists in a continuum.
On one end of the spectrum you have client computers running a "thick" operating system (like Windows or a desktop Linux distribution) and connecting via client software to hosted applications (via RemoteApp shortcuts and the Remote Desktop Protocol (RDP), or via Citrix ICA protocol).
In the middle of the spectrum you have clients connecting via these same protocols to full-blown desktop sessions (rather than a single application), but using a shared operating system installation. This is typically the world of Windows "Terminal Services".
On the far end of the spectrum you have what's typically known as a Virtual Desktop Infrastructure (VDI) where client devices are very stripped down and only host client software to connect to a hosted operating system instance.
All of these situations are physically feasible, but you'd do yourself a favor to start investigating the licensing costs before you go down the road of spec'ing servers, etc.
The licensing costs in the Microsoft world include either Terminal Services Client Access Licenses or Windows Virtual Enterprise Centralized Desktop (VECD) licenses of operating systems to contend with for each device or user accessing the VDI solution. Licensing for your desktop application software, depending on where on the spectrum you're falling, may also be different than you currently use and this necessitate additional license purchases.
It's likely that you're going to find that the acquisition costs of a VDI infrastructure are similiar, if not more expensive, than going down the traditional "thick client" route. Phyisically and pratically using thin-client devices sounds like a "win", but software licensing expense has traditionally more than made up for any hardware cost savings, which leaves only "soft cost" management and TCO savings as justification.
Edit:
Ryan Bolger hit it right on the head with his answer (and I +1'd him) with respect to "soft cost" savings, which you're right to identify as the place to save money.
Learning how to centrally deploy software, manage user environments, and generally maintain the hell out of your network using Group Policy will build your personal knowledge of the "innards" and operation of a Windows network and will have far fewer "moving parts" than a VDI infrastructure. Even if you had a VDI infrastructure, frankly, I think you'd still be able to leverage immense benefits from Group Policy-fu.
VDI and remote application delivery is a great solution for very task-specific application, or delivery of applications over slow or unreliable network connections (think "shared Microsoft Access database over a T1-based WAN"). I don't think that desktop virtualization, at least in the current incarnation as an excessive-licensing-fee-based minefield, is "the answer".
I'll even jump out on a limb and say that, with proper "care and feeding" maintenance of very large fleets of client computers running Windows isn't really all that hard, using the built-in tools in Windows Server, WSUS, good knowledge of scripting, and an understanding of how Windows itself and your application software works. Automating your client computer build, removing users' Administrator rights, and getting a handle on your OS and application update deployment infrastructure will take you leaps and bounds ahead.
Two ways.
PowerShell Module:
From a Windows 8/Windows Server 2012 machine in PowerShell you should be able to use the Get-RDPersonalVirtualDesktopAssignment command. You can specify a connection broker to connect to with -ConnectionBroker <String>, or query by collection and user.
Active Directory:
You may be able to find the information in Active Directory if Windows Server 2012 maintains the same schema as in 2008 R2, you may be able to find it by querying users for the msTSPrimaryDesktop attribute. To search by attribute in PowerShell, on Windows 7/2008 R2 and previous versions you may need to first run Import-Module ActiveDirectory, and then execute the command:
Get-ADUser -Properties msTSPrimaryDesktop -Filter { msTSPrimaryDesktop -like "*" }
The msTSPrimaryDesktop property is used in VDI in Windows Server 2008 R2 at least, and is also accessible on certain machines via Active Directory Users and Computers (dsa.msc). I am not sure if Windows Server 2012 uses this property as a user's primary desktop may be relative to a particular collection, not global as per 2008 R2.
Best Answer
You're going to need to do a proper cost/benefit analysis on this, tailored to your specific scenario.
The main advantage of implementing a VDI solution is for situations where you do precisely the same thing on a lot of workstations for a lot of users, and can connect them all back to a single central system.
In your case, I'd look very closely at:
IMO the VDI/Thin client architecture still doesn't stack up as particularly cost-effective, and won't unless the cost of thin client hardware drops significantly. The main benefit is for situations (e.g. Point of sale) where you want to keep the operation of the system as close to the centre as possible).
One option I often float and implement cheaply/effectively is to setup a terminal services environment that serves up the core business apps to any and all clients - This ignores the stuff that people like to customize (development environments etc) but standardizes access for your 'cookie cutter' applications (e.g. timesheet system, payroll). This, and/or shifting some stuff out to SaaS applications (e.g. openair.com) can cut down the amount of extra hassle you have to go through for giving VPN users access (you just let them reach the TS), and less time spent configuring each PC with all the fiddly apps.