In which scenarios should I consider a functional programming languages better suited to do a given task? Besides the so recently popular multicore problem of parallel programming.
Anything that involves creating sequence of derived data elements using a number of transformation steps.
Essentially, the "spreadsheet problem". You have some initial data and set of row-by-row calculations to apply to that data.
Our production applications do a number of statistical summaries of data; this is all best approached functionally.
One common thing we do is a match-merge between three monstrous data sets. Similar to a SQL join, but not as generalized. This is followed by a number of calculations of derived data. This is all just functional transformations.
The application is written in Python, but is written in a functional style using generator functions and immutable named tuples. It's a composition of lower-level functions.
Here's a concrete example of a functional composition.
for line in ( l.split(":") for l in ( l.strip() for l in someFile ) ):
print line[0], line[3]
This is one way that functional programming influences languages like Python.
Sometimes this kind of thing gets written as:
cleaned = ( l.strip() for l in someFile )
split = ( l.split(":") for l in cleaned )
for line in split:
print line[0], line[3]
If I decided to switch to a functional programming language which do you consider are the biggest pitfalls that I will face? (Besides the paradigm change and the difficulty to evaluate performance due to lazy evaluation).
Immutable objects is the toughest hurdle.
Often you'll wind up calculating values that create new objects instead of updating existing objects. The idea that it's a mutable attribute of an object is a hard mental habit to break.
A derived property or method function is a better approach. Stateful objects are a hard habit to break.
With so many functional programming languages out there, how would you choose the one the better suit your needs?
It doesn't matter at first. Pick any language to learn. Once you know something, you're in a position consider picking another to better suit your needs.
I've read up on Haskell just to understand the things Python lacks.
Best Answer
Tongue slightly in cheek, I'd answer: Lisp, Lisp, and Lisp.
For question number one, this is probably a controversial answer. People who are comfortable with the Microsoft development stack would, of course, consider F# to be more mainstream - after all, it's right there in your developer's tool chain, ready to be installed (or already there) on millions upon millions of PCs. The academic community is more excited about Haskell, but here also real-world usage is less than they would like to achieve. Telecom engineers swear by Erlang instead. But when taking all dialects together - Scheme for teaching and extension languages, Common Lisp for enterprise-size applications, Emacs lisp for the one true editor, Clojure for the cool young Java kids... they might just be the most-used predominantly functional language out there. (Note that programming language market share is an intensely politicized topic, and any answer you receive will be ripped to shreds by other opinions.)
Question two is easy. Unless you want to count the purely mathematical lambda calculus, Lisp was first - although it, too, was initially intended just as a mathematical modelling tool, it turned out to be easily implementable, and the rest is history. That was in the 1950s.
Question three: Common Lisp is decidedly non-pure, while Scheme more closely resembles pure axiomatic systems like Haskell, which rigorously separate side effects from computation. Again, questions about market share are always controversial, and Scheme is certainly not the purest FP language, just (I guess) the most used - even if much of this is due to cohorts of computer science freshmen. And Common Lisp is not the least-pure language either - I think all of the the various "Lisp on the JVM" tools are less pure, since leveraging the huge Java library ecosystem is one of their reasons for existence. If I were forced to guess, I'd say one of them will probably the next big winner in functional programming.