In which scenarios should I consider a functional programming languages better suited to do a given task? Besides the so recently popular multicore problem of parallel programming.
Anything that involves creating sequence of derived data elements using a number of transformation steps.
Essentially, the "spreadsheet problem". You have some initial data and set of row-by-row calculations to apply to that data.
Our production applications do a number of statistical summaries of data; this is all best approached functionally.
One common thing we do is a match-merge between three monstrous data sets. Similar to a SQL join, but not as generalized. This is followed by a number of calculations of derived data. This is all just functional transformations.
The application is written in Python, but is written in a functional style using generator functions and immutable named tuples. It's a composition of lower-level functions.
Here's a concrete example of a functional composition.
for line in ( l.split(":") for l in ( l.strip() for l in someFile ) ):
print line[0], line[3]
This is one way that functional programming influences languages like Python.
Sometimes this kind of thing gets written as:
cleaned = ( l.strip() for l in someFile )
split = ( l.split(":") for l in cleaned )
for line in split:
print line[0], line[3]
If I decided to switch to a functional programming language which do you consider are the biggest pitfalls that I will face? (Besides the paradigm change and the difficulty to evaluate performance due to lazy evaluation).
Immutable objects is the toughest hurdle.
Often you'll wind up calculating values that create new objects instead of updating existing objects. The idea that it's a mutable attribute of an object is a hard mental habit to break.
A derived property or method function is a better approach. Stateful objects are a hard habit to break.
With so many functional programming languages out there, how would you choose the one the better suit your needs?
It doesn't matter at first. Pick any language to learn. Once you know something, you're in a position consider picking another to better suit your needs.
I've read up on Haskell just to understand the things Python lacks.
There's certainly a noticeable trend towards functional programming, or at least certain aspects of it. Some of the popular languages that at some point adopted anonymous functions are C++ (C++11), PHP (PHP 5.3.0), C# (C# v2.0), Delphi (since 2009), Objective C (blocks) while Java 8 will bring support for lambdas to the language . And there are popular languages that are generally not considered functional but supported anonymous functions from the start, or at least early on, the shining example being JavaScript.
As with all trends, trying to look for a single event that sparked them is probably a waste of time, it's usually a combination of factors, most of which aren't quantifiable. Practical Common Lisp, published in 2005, may have played an important role in bringing new attention to Lisp as a practical language, as for quite some time Lisp was mostly a language you'd meet in an academic setting, or very specific niche markets. JavaScript's popularity may have also played an important role in bringing new attention to anonymous functions, as munificent explains in his answer.
Other than the adoption of functional concepts from multi-purpose languages, there's also a noticeable shift towards functional (or mostly functional) languages. Languages like Erlang (1986), Haskell (1990), OCaml (1996), Scala (2003), F# (2005), Clojure (2007), and even domain specific languages like R (1993) seem to have gained a strong following strongly after they were introduced. The general trend has brought new attention to older functional languages, like Scheme (1975), and obviously Common Lisp.
I think the single more important event is the adoption of functional programming in the industry. I have absolutely no idea why that didn't use to be the case, but it seems to me that at some point during the early and mid 90s functional programming started to find it's place in the industry, starting (perhaps) with Erlang's proliferation in telecommunications and Haskell's adoption in aerospace and hardware design.
Joel Spolsky has written a very interesting blog post, The Perils of JavaSchools, where he argues against the (then) trend of universities to favour Java over other, perhaps more difficult to learn languages. Although the blog post has little to do with functional programming, it identifies a key issue:
Therein lies the debate. Years of whinging by lazy CS undergrads like me, combined with complaints from industry about how few CS majors are graduating from American universities, have taken a toll, and in the last decade a large number of otherwise perfectly good schools have gone 100% Java. It's hip, the recruiters who use "grep" to evaluate resumes seem to like it, and, best of all, there's nothing hard enough about Java to really weed out the programmers without the part of the brain that does pointers or recursion, so the drop-out rates are lower, and the computer science departments have more students, and bigger budgets, and all is well.
I still remember how much I hated Lisp, when I first met her during my college years. It's definitely a harsh mistress, and it's not a language where you can be immediately productive (well, at least I couldn't). Compared to Lisp, Haskell (for example) is a lot friendlier, you can be productive without that much effort and without feeling like a complete idiot, and that might also be an important factor in the shift towards functional programming.
All in all, this is a good thing. Several multi-purpose languages are adopting concepts of paradigm that might have seemed arcane to most of their users before, and the gap between the main paradigms is narrowing.
Related questions:
Further reading:
Best Answer
C#, VB.NET, Python, JavaScript, now even C++0x provides first-class functions.
Update:
An implementation of closures requires lambda-lifting - a technique originally quite alien for the imperative programmers. Proper first-class functions (which includes closures) requires at least some support from the runtime and VM. It also took some time to adopt the old functional techniques into the imperative world.
And, the most important part - first-class functions are barely usable if there is no garbage collection present. It was introduced into a mass programming only recently, with Java and, consequently, .NET. And the original Java approach was to oversimplify the language in order to keep it comprehandable by the average coders. .NET first followed, and only recently parted from that (IMHO, quite justified and appropriate) direction.
All such concepts takes time to be digested by the industry.