Functional programming (in javascript and python): Part 1: Why?

The problem

At the beginning of this year I started thinking how would I describe what I did as a programmer for the last 13 years in just a few words, and this is the conclusion:

List processing and debugging. 

Now let me explain: If I abstract enough, everything looks like a list: arrays, dictionaries even objects, considering them a list of properties with values. Complex objects become lists of lists. They're everywhere: from databases where these days you get back lists of rows/objects, to models, to the UI for instance $("...") in jQuery. So basically all day long I wrote functions and methods to process these lists and lists of lists, transforming them, changing them, persisting and loading them, send and receive them through the networks and internet or displaying them on the screen.

The big problem is that if the code becomes a little complex, you expect at the end of some processing some result and it's not there. Basically you expect the system to be in a state, and it is in another, thus having an error or a bug, so you start following what happens to see why. If the system uses events and async operations your debugging work starts to take a lot of time and effort. Thinking about time, I would correct the conclusion above to

30% list processing, 70% debugging

The causes

For years I looked for solutions: from OOP, to patterns, to separation of concerns, agile methodologies practices such as unit testing and test driven development. All helped, but none seem to address the biggest issues: state is everywhere thus unexpected state changes, not to mention concurrent programming.

The solution

Then I remembered about LISP (List programming), and how strange it seemed. But somehow it seems to do exactly what I needed, unfortunately not in the programming languages that I needed, but then I started to realise that the languages I worked with are multi paradigm and yes functional programming concepts are embedded right in (javascript/python).

So basically what functional programming addresses is: list processing using 3 main functions: map, reduce and filter. Using these 3 functions, simpler data structures like arrays, dictionaries and lists instead of objects used as data structures and higher order functions, the code to process lists becomes simpler and smaller.

Example

Let's see the total of an order in the shopping cart, for groceries:

Python, using imperative programming and OOP (badly, like pretty much everyone):

class Order()
     orderItems = []
     def __init__():
           pass
...

class OrderItem()
     name = None
     quantity = 0.0
     price = 0.0

     def __init__(name=None, quantity=0.0, price=0.0):
           self.name=name

           self.quantity =quantity

           self.price=price

...

order = Order()
order.orderItems.append(OrderItem(name="", quantity=1,price=0.99))

order.orderItems.append(OrderItem(name="", quantity=1,price=0.99))

def is_grocery(item)
     return ...

sum = 0
for item in order.orderItems
     if is_grocery(item):
         sum+=item.quantity*item.price

print sum

Python, using FP:

order = {"orderItems":[
          { "name":"sada""quantity":2,"price":0.99},
          {"name":"zjdfj""quantity":1,"price":2.99},

          ...]}

def is_grocery(item):
      return ...

print reduce(
        lambda total1,total2: total1+total2,
        map(
            lambda orderItem: i1["quantity"]*i1["price"]
            filter(
                 lambda orderItem:is_grocery(orderItem),
                 order["orderItems"]
            )
        )

About state changes, we'll speak in the next article.

Conclusion

So why didn't functional programming replace imperative programming since it has many areas where it clearly is better (maybe not in the example above)? Because it is much more difficult to understand and use, even though the resulting code is smaller. If you look at the code above you see that is starts the other way round: it wants to "reduce" 2 totals for order items and then you see that the reduce is applied on a list of totals for order items which is applied on a filtered list of items that are groceries. This is the reason people say functional programming shows what the code does not how.

Hierarchical persistent data structures in javascript (using underscore.js)

Lately, I have been studying a lot about functional programming, and things like LISP, Clojure, Scala but since mostly I program in Javascript and python, I also studied a lot the functional programming concepts in these 2 languages.

The problem

I have a model (using MVC pattern) and I want to keep both the initial model state, and the one that is changed afar some operations. I wanted initially to use something like cloning or deep copying but it seems just to waste memory, if for instance I change only one thing in a hierarchy, so I don't need everything clones.

The solution : persistent data structures

I knew that what I wanted is to use persistent data structures in javascript and although they are available in libraries such as immutable or mori, I needed one that is hierarchical.

Persistent data sturctures acording to wikipedia:

In computing, a persistent data structure is a data structure that always preserves the previous version of itself when it is modified. Such data structures are effectively immutable, as their operations do not (visibly) update the structure in-place, but instead always yield a new updated structure. (A persistent data structure is not a data structure committed to persistent storage, such as a disk; this is a different and unrelated sense of the word "persistent.")

Solution:

So I want something like:

  initial_state = {a:1, b:2};  
  modified_state = change(model,"a","abc"))   

where initial_state remains unchanged and modified_state is {a:"abc",b:2}. This is very simple using something like _.clone(initial_state) in underscore.js. But what if I want something like:

  initial_state = model = {l:[{id:1}, {id:2}], b:2};  
  modified_state = change(model,"l[1].id",3)  

resulting in: modified_state = {l:{id:1},{id:3}], b:2}. Here copy, deep copy or clone doesn't help. So what if I write a function like this:

 function change(obj,path,val)  
 {  
   var new_obj = _.clone(obj);  
   var path = path.replace("[",".").replace("]","");  
   var levels = path.split(".");  
   if(levels.length>1)  
   {  
     var head = _.first(levels)  
     var tail = path.replace(head+".","")  
     new_obj[head] = change(obj[head],tail,val)  
   }  
   else  
   {  
     new_obj[path] = val;  
   }  
   return new_obj;  
 }  

It works!

I can even do:

 initial_state = {g:  
     [  
       {id:1, values:[1,2,3]},  
       {id:2, values:[3,4]}  
     ],  
   b:2}  
 new_state1 = change(model,"g[1].values[1]",7);  
 new_state2=change(model,"g[1].values",[7,2]);