I think you've got a point there, but cp, rm, cd and a lot of others change state, so they aren't really functions. The UNIX philosophy is more about doing only one thing but doing it well; often doing it well means allowing functional usage, but not always.
In which scenarios should I consider a functional programming languages better suited to do a given task? Besides the so recently popular multicore problem of parallel programming.
Anything that involves creating sequence of derived data elements using a number of transformation steps.
Essentially, the "spreadsheet problem". You have some initial data and set of row-by-row calculations to apply to that data.
Our production applications do a number of statistical summaries of data; this is all best approached functionally.
One common thing we do is a match-merge between three monstrous data sets. Similar to a SQL join, but not as generalized. This is followed by a number of calculations of derived data. This is all just functional transformations.
The application is written in Python, but is written in a functional style using generator functions and immutable named tuples. It's a composition of lower-level functions.
Here's a concrete example of a functional composition.
for line in ( l.split(":") for l in ( l.strip() for l in someFile ) ):
print line[0], line[3]
This is one way that functional programming influences languages like Python.
Sometimes this kind of thing gets written as:
cleaned = ( l.strip() for l in someFile )
split = ( l.split(":") for l in cleaned )
for line in split:
print line[0], line[3]
If I decided to switch to a functional programming language which do you consider are the biggest pitfalls that I will face? (Besides the paradigm change and the difficulty to evaluate performance due to lazy evaluation).
Immutable objects is the toughest hurdle.
Often you'll wind up calculating values that create new objects instead of updating existing objects. The idea that it's a mutable attribute of an object is a hard mental habit to break.
A derived property or method function is a better approach. Stateful objects are a hard habit to break.
With so many functional programming languages out there, how would you choose the one the better suit your needs?
It doesn't matter at first. Pick any language to learn. Once you know something, you're in a position consider picking another to better suit your needs.
I've read up on Haskell just to understand the things Python lacks.
Best Answer
The only correct answer to this is "sometimes". There are a lot of tricks that functional languages can use to avoid wasting memory. Immutability makes it easier to share data between functions, and even between data structures, since the compiler can guarantee that the data won't be modified. Functional languages tend to encourage the use of data structures that can be used efficiently as immutable structures (for instance, trees instead of hash tables). If you add laziness into the mix, like many functional languages do, that adds new ways to save memory (it also adds new ways of wasting memory, but I'm not going to go into that).