Functional Programming – Are Closures Sufficient to Characterize It?

In which scenarios should I consider a functional programming languages better suited to do a given task? Besides the so recently popular multicore problem of parallel programming.

Anything that involves creating sequence of derived data elements using a number of transformation steps.

Essentially, the "spreadsheet problem". You have some initial data and set of row-by-row calculations to apply to that data.

Our production applications do a number of statistical summaries of data; this is all best approached functionally.

One common thing we do is a match-merge between three monstrous data sets. Similar to a SQL join, but not as generalized. This is followed by a number of calculations of derived data. This is all just functional transformations.

The application is written in Python, but is written in a functional style using generator functions and immutable named tuples. It's a composition of lower-level functions.

Here's a concrete example of a functional composition.

for line in ( l.split(":") for l in ( l.strip() for l in someFile ) ):
    print line[0], line[3]

This is one way that functional programming influences languages like Python.

Sometimes this kind of thing gets written as:

cleaned = ( l.strip() for l in someFile )
split = ( l.split(":") for l in cleaned )
for line in split:
     print line[0], line[3]

If I decided to switch to a functional programming language which do you consider are the biggest pitfalls that I will face? (Besides the paradigm change and the difficulty to evaluate performance due to lazy evaluation).

Immutable objects is the toughest hurdle.

Often you'll wind up calculating values that create new objects instead of updating existing objects. The idea that it's a mutable attribute of an object is a hard mental habit to break.

A derived property or method function is a better approach. Stateful objects are a hard habit to break.

With so many functional programming languages out there, how would you choose the one the better suit your needs?

It doesn't matter at first. Pick any language to learn. Once you know something, you're in a position consider picking another to better suit your needs.

I've read up on Haskell just to understand the things Python lacks.

Notwithstanding any specific ideas on the part of language designers, it bears mentioning that authors and stewards of programming languages are, in the end, pushing a product. So, I might ask why I'd want a camera-phone when my plain phone is a better phone and my camera a better camera, but that isn't going to stop manufacturers of both devices from trying to broaden their product's offering to attract new customers.

Once you look at it from that perspective, then notions of preserving the integrity of the original language become a matter of degrees and tradeoffs. If I'm the author of OOP language AwesomeCode and I see people starting to get interested in new functional language FCode, do I tell my users "sorry, but this is an OOP language only" and risk them going to C# instead to get at its lambas, or do I cave and grudgingly include some of FCode's functionality?