Are you using Windows, OS X or some version of Linux? Are you using the Haskell Platform? Have you had a version of ghc or cabal before? For a Linux distribution, subtleties about your package manager may come in, of course. (Traces of an old ghc in particular, and an old ~/.ghc/ directory can be a source of trouble.)
Here are a few elementary thoughts of the type one goes through on #haskell with such problems. (My comprehension is not completely adequate, of course):
The chief question seems to be, Why you were being invited to do what should be local installs with sudo? A global install (cabal install pony --global) would of course require privileges if ghc and its libraries are in /usr/... or some other protected place, but otherwise sudo vs non-sudo is independent of the place of installation. What you do with cabal install pony --user (--user is the default, in theory) should not require superuser authority. (I have sometimes found on OS X that privileges are requested where the gcc needs to be called, but this has usually been due to curiosities about my setup.) But in any case sudo doesn't affect where cabal is putting them: the implicit --user and explicit --global, and more specific incantations for development, do that.
If you do ghc-pkg list, for example, it will divide the packages into the different places they are registered in according to two or more package.conf.d directories it is summarizing. On my laptop at the moment these are
/Users/applicative/.ghc/x86_64-darwin-7.0.3/package.conf.d/...
for the local things in ~/.cabal/lib/... and the protected
/Library/Frameworks/GHC.framework/Versions/7.0.3-x86_64/usr/lib/ghc-7.0.3/package.conf.d
for things that were installed globally with the Haskell Platform installer (this location involves some OS X peculiarities, ghc, ghci and so on are in the woods somewhere, but symlinked to /usr/bin). The conf files for different packages tell you exactly where the libraries were installed. So, for example about the sacred base library,
$ cat base-4.3.1.0-f5c465200a37a65ca26c5c6c600f6c76.conf
tells me:
import-dirs:
/Library/Frameworks/GHC.framework/Versions/7.0.3-x86_64/usr/lib/ghc-7.0.3/base-4.3.1.0
library-dirs:
/Library/Frameworks/GHC.framework/Versions/7.0.3-x86_64/usr/lib/ghc-7.0.3/base-4.3.1.0
In any case, where does ghc-pkg list say your cabal install-ed packages are going? In the ~/.cabal folder, look at the file config. If you haven't edited it, I think the commented and uncommented lines, if they state a preference, are stating the defaults for installation with --global and --user. In the ~.ghc/ directory check out the subdirectory myghcversion/package.conf.d and see if anything is there, which should be the same as what ghc-pkg tells you. (You might study the options for ghc-pkg in general, eg. ghc-pkg check and ghc-pkg recache, if you haven't. You may have installed something in some odd way.)
If you installed ghc and cabal and co. by installing the Haskell Platform with a binary installer or your package manager, which seems like a good idea, it is also a good idea, I think, to keep the Platform libraries as something sacred, and make sure you never install anything globally from Hackage; among other things this is likely to have you overwriting Platform libraries -- though this doesn't seem the difficulty here: it would be more obvious if it were.
Best Answer
You can delete the files yourself from the packages directory. However, the reason no command to do so is provided is there's in general no guarantee something may not have linked against them elsewhere, and so such deletions may cause breakages. That said, there's also a tool that goes and does the deletion for you if you really want it.
http://hackage.haskell.org/package/cabal-uninstall
And there's a tool with a bit more functionality that also lets you figure out what packages have no reverse deps, so at least no other packages break:
https://github.com/iquiw/cabal-delete