How to solve it? Complexity in code - Improving flow control through functional pipelines

Code: https://gitlab.com/danbunea/improving-control-flow-in-code-using-functional-pipelines

Why?

You:

Why?

Me:

Because code is better...

You:

Code is better? How?

Me:

Because is it easier to: 

• read
• extend
• debug

You:

Prove it!

Me:

Ok let's look at the following problem:

In practice

We have to write an endpoint which returns an offer by its id:

GET /offer-by-id/:offer-id
Possible results:

- 200 {"offer-id":2, "offer-data":""}
- 400 {"errors":["The id you provided is invalid"}]}
- 404 {"errors":["The id you provided cannot be found"}]}
IMPORTANT: if an id is valid but not found the deposit must be notified!

Solution 1: IFs

The problem could be solved by:

Or in code:

Whenever we have ifs in code, it becomes dificult to read so maybe we could simplify it, making it more like:

step 1
then step 2
then step 3

Or more like

validate
then find offer
then jsonify

Hmm, can we?

Solution 2: Exceptions

We'll use exceptions to break out flow:



The solution above is very present in Java, even if the way the errors are caught might not be this explicit.

Because it's a fairly simple example we could also use the strategy pattern. Basically we use ifs to choose a strategy, then we execute it to give us the result. 

Solution 3: strategy pattern

Functional programming gives us sever possibilities using pipes. 

What is a pipe?

A pipe makes sure that steps get executed in a certain order and that the result of a step is passed to the next step. Like:

In our case we'd like something like:

validate(requestId)
.then(findOfferById)
.then(jsonify)

Solution 4 Functional either or railway oriented programming

This is a very common solution in typed functional languages such as F# or Haskel, but it's becoming very common in Java as well.

Having a single pipeline is very beautiful but how do we handle errors? By using two parallel pipes. A pipe for the happy path and a path for the errors. Basically all our functions can return either a SuccesfullResponse or a ErrorResponse. This will get passed on to the next function that will process it and return again either a SuccesfullResponse or a ErrorResponse.

In F# we'd have something like:

findOrderById: IResponse -> SuccesfullResponse | ErrorResponse

we might describe it like:

SuccesfullResponse | ErrorResponse aFunction(SuccesfullResponse | ErrorResponse response)

For the happy path we'd have:

And if we get a validation error in the first step we'll have:

And the code, will have a class for Success and a class for Error. Each will inherit an IResponse and will implement two functions receiving a function (lambda) then and fail. In Succes we'll return the result of applying the function on what we have on then and the data we have on fail, and on Error we'll do exactly the opposite. 

Imaging you'd replace then with map and fail with orElseGet, doesn't that sound like an optional?



In dynamic languages, there are other options. But first let's describe functional composition. 

Functional composition is when you combine 2 (or more) functions into one, then apply it. It's pretty much like pipe but you may do the composition at runtime. 

Now we can look at the two options. First is:

Solution 5: pipeline with flag

We'll pass through the pipe an object that contains a flag which tells you whether there were errors before. Basically using a value in your data instead of using the type of the data (has response property instead of type: Successful or Error).

In our case, the flag is whether a response has been set already:

if(state.response) return state;

Solution 6: pipeline + overflow pipeline 

The second option is to have a pipeline and an overflow pipeline.  

 We'll use exceptions once again to bypass the normal pipeline and go to the overflow pipeline.

Unlike the previous two options when an exception happend it will jump straight to the end, bypassing the next steps directly.



The code: https://gitlab.com/danbunea/improving-control-flow-in-code-using-functional-pipelines/tree/stage-1

Readability

In terms of readability it could be a lot easier, to see the code as a pipeline:

step 1
then step 2
then step 3

While also handling the errors:

step 1
then step 2
then step 3
fail on-error

or 

safe(
   step 1
   then step 2
   then step 3
)

Extensibility

But what about extensibility?

We now have to modify our endpoint

1. to check if the offers are still active. If the aren't we need to return an error
2. to update the number of times the offer has been accessed

GET /offer-by-id/:offer-id

Possible results:

- 200 {"offer-id":2, "offer-data":"", "active":true, "requests":1}
- 400 {"errors":["The id you provided is invalid"}]}
- 400 {"errors":["The offer expired"}]}
- 404 {"errors":["The id you provided cannot be found"}]}

IMPORTANT: if an id is valid but the offer expired the deposit must be notified!

We changed the tests:

Then we change the code and we can look at how
For solution 1:

The way we solved it was to add more ifs inside an existing if thus increasing the cyclomatic complexity of the solution making it even harder to read. And real life code tends to be more complex than this.


For solution 2:

We did:

• added a new exception
• changed the code inside the try catch block
• changed the code in the catch

For solution 4:

What did we do:

• added two new functions, completely independent 
• modified an existing one
• added steps to the pipeline

For solution 5:

What did we do:

• added a new function
• heavily modified an existing one
• added a step to the pipeline

For solution 6:

What did we do:

• added a two new functions
• added them as steps to the pipeline

No existing code modified! (except for the pipeline, which is expected)

The code: https://gitlab.com/danbunea/improving-control-flow-in-code-using-functional-pipelines/tree/stage-2

The merge request in gitlab: https://gitlab.com/danbunea/improving-control-flow-in-code-using-functional-pipelines/merge_requests/3/diffs#faff669626dfa73714964353a02a5101dce1b3a7

Debugging

Let's say we need to start logging what is going on. For ifs we may end up with:

Not exactly easy to know where to insert the logging. 
But for a pipeline, we just need to insert some tracing function between the steps:

giving us:

or

And the log is like:

Last, let's see where pipelines could be used.

Frontend/Javascript with Promises:

or front end Clojurescript:

or backend Python:

or Java pipelines

or

The example comes from the book "Functional Style" by our colleague Richard Wild  https://functional.works-hub.com/learn/the-functional-style-part-5-higher-order-functions-i-function-composition-and-the-monad-pattern-bc74a?utm_source=blog&utm_medium=blog&utm_campaign=j.kaplan

Conclusion

Typed languages (including Java) you should use Either.

Dynamic languages the best would be pipeline with exceptions.

Thinking in Reagent as a reaction to Thinking in React. A 1:1 transformation from Javascript/React to ClojureScript/Reagent

This article: https://reactjs.org/docs/thinking-in-react.html introduces people to working with React in Javascript. I thought it would be nice to have the same simple application made in Clojurescript with Reagent (which also uses React underneath)

For example

class ProductCategoryRow extends React.Component {

  render() {

    const category = this.props.category;

    return (

     

       

          {category}

       

     

    );

  }

}

In Reagent:

(defn product-category-row-component [category]
  [:tr
   [:th
    {:col-span 2}
    category]])


 I made an interactive presentation and I put it here:

http://www.danbunea.ro/presentations/thinking%20in%20reagent.html

What is wrong with static typing in JavaScript and how clojure.spec solves the problem (Part 2)

The problem

The main problem with dynamic typing seems to be fear that the wrong type of data will end up in the wrong place (function input for instance).

However it turns out static typing is pretty useless.

Example 1: Simple types Age

Let's say you have to record an age for a person in a variable, or have it as a parameter in a function.

You would do something like:

int age = 25;

or

function something(int age...)

Someone pretended even that static typing shows intent. Now the only thing that is in here, is that it will be an int. There is nothing protecting the age from being either negative (-2) or too big (4000, it might work if you're talking about the age of the pyramids). So it is not intent, it is just int, not further protection, so pretty much useless.

Solution 1

In Clojure REPL using spec (require '[clojure.spec :as s]) we define a spec saying we want a natural int (positive int) and it should be smaller then let's say 150:

(s/def ::age (s/and nat-int? (fn [x] (< x 150))))

Now:

user=> (s/valid? ::age "a")

false

user=> (s/valid? ::age -12)

false

user=> (s/valid? ::age true)

false

user=> (s/valid? ::age 1.21)

false

user=> (s/valid? ::age 4000)

false

user=> (s/valid? ::age -2)

false

user=> (s/valid? ::age 0)

true

user=> (s/valid? ::age 12)

true

user=> (s/valid? ::age 29)

true

user=> (s/valid? ::age 99)

true

Example 2: Composed types: Person

Let's say we get through a web call a json like:

{
  "id":6,
  "name":"Dan",
  "age":28
}

Usually people would create a class

class Person
{
    int id;
    string name;
    int age;
}

So we have the same problems, for instance name might be null, or age might be negative.

Then in modern apps, you get json and you send json, so you need to be able to serialize and deserialize to json this class. What happens if one of the parameters is not comform or missing?

Solution 2

(s/def ::id nat-int?)

(s/def ::name string?)

(s/def ::person (s/keys :req-un [::id ::name ::age]))

Now:

user=> (s/valid? ::person {:id 1, :name "Adi"})

false

user=> (s/valid? ::person {:id 1, :name "Adi" :age -1})

false

user=> (s/valid? ::person {:id 1, :name "Adi" :age 40})

true

What's even cooler, is that if it isn't valid, you can get an explanation:

user=> (s/explain ::person {:id 1, :name "Adi" :age -1})

In: [:age] val: -1 fails spec: :user/age at: [:age] predicate: nat-int?

user=> (s/explain ::person {:id 1})

val: {:id 1} fails spec: :user/person predicate: (contains? % :name)

val: {:id 1} fails spec: :user/person predicate: (contains? % :age)

Example 3: Hierarchies 

What if you have:

{
    "id": 6,
    "name": "Dan",
    "age": 28,
    "children": [{
            "id": 7,
            "name": "Alex",
            "age": 5
        }
    ]
}

When the first object is a parent, in a school and he must have at least one child? You can enforce the relationship by writing a function and in the constructor, but then you also need to change the serialization/deserialization from json to enforce the rules, and of course you will write more code and you will forget to check it once, and there will be a bug.

And the most common problem of our times. The json is like:

{
    "id": 11,
    "name": "Maria",
    "age": 95,
    "children": [{
                "id": 5,
                "name": "Elena",
                "age": 28,
                "children": [{
                    "id": 6,
                    "name": "Dan",
                    "age": 60,
                    "children": [{
                        "id": 7,
                        "name": "Alex",
                        "age": 5
                    }],
                    {
                        "id": 9,
                        "name": "Alina",
                        "age": 32,
                        "children": [{
                            "id": 121,
                            "name": "Luiza",
                            "age": 0
                        }]
                    }
                }],

                {
                    "id": 23,
                    "name": "Petru",
                    "age": 70,
                    "children": [{
                            "id": 4,
                            "name": "Adrian",
                            "children": [{
                                "id": 45,
                                "name": "Denis",
                                "age": 12
                            }],
                        ]
                    }]
            }]
}

You have a single error but where? (Maria / Petru / Adrian - missing "age"). It is not only hard to validate it but it is hard to show explicitly where the error occurred.

Solutions

(s/+ says that there will be a collection of person's with a minimum of 1:

(s/def ::children (s/+ ::person))

(s/def ::parent (s/keys :req-un [::id ::name ::age ::children]))

Now:

user=> (s/valid? ::parent {:id 1, :name "Adi" :age 40 :children []})

false

user=> (s/valid? ::parent {:id 1, :name "Adi" :age 40 :children [{:id 1, :name "Adi" :age 40}]})

true

user=> (s/valid? ::parent {:id 1, :name "Adi" :age 40 :children [{:id 1, :name "Adi" :age 40}, {:id 2, :name "Dan" :age 20}]})

true

and if we don't have children:

(s/explain ::parent {:id 1, :name "Adi" :age 40 :children []})

In: [:children] val: () fails spec: :user/person at: [:children] predicate: :user/person,  Insufficient input

Even cooler is that you can check relations between data, like if the children are younger then their parents: 

(defn parent-older-than-children? [parent] (reduce #(or %1 %2) (map #(> (:age parent) (:age %)) (:children parent))))

we redefine the ::parent

(s/def ::parent (s/and (s/keys :req-un [::id ::name ::age ::children]) parent-older-than-children?))

user=> (s/valid? ::parent {:id 1, :name "Adi" :age 40 :children [{:id 1, :name "Adi" :age 50}]})

false

user=> (s/explain ::parent {:id 1, :name "Adi" :age 40 :children [{:id 1, :name "Adi" :age 50}]})

val: {:id 1, :name "Adi", :age 40, :children [{:id 1, :name "Adi", :age 50}]} fails spec: :user/parent predicate: parent-older-than-children?

user=> (s/valid? ::parent {:id 1, :name "Adi" :age 45 :children [{:id 1, :name "Adi" :age 25}, {:id 2, :name "Dan" :age 20}]})

true

In Part 3 we will look at functions and one more thing ...

What is wrong with static typing in JavaScript and how clojure.spec solves the problem (Part 1)

The problem

The main problem with dynamic typing seems to be fear that the wrong type of data will end up in the wrong place (function input for instance).

However it turns out static typing is pretty useless.

Example 1: Simple types Age

Let's say you have to record an age for a person in a variable, or have it as a parameter in a function.

You would do something like:

int age = 25;

or

function something(int age...)

Someone pretended even that static typing shows intent. Now the only thing that is in here, is that it will be an int. There is nothing protecting the age from being either negative (-2) or too big (4000, it might work if you're talking about the age of the pyramids). So it is not intent, it is just int, not further protection, so pretty much useless.

Example 2: Composed types: Person

Let's say we get through a web call a json like:

{
  "id":6,
  "name":"Dan",
  "age":28
}

Usually people would create a class

class Person
{
    int id;
    string name;
    int age;
}

So we have the same problems, for instance name might be null, or age might be negative.

Then in modern apps, you get json and you send json, so you need to be able to serialize and deserialize to json this class. What happens if one of the parameters is not comform or missing?

Example 3: Hierarchies 

What if you have:

{
    "id": 6,
    "name": "Dan",
    "age": 28,
    "children": [{
            "id": 7,
            "name": "Alex",
            "age": 5
        }
    ]
}

When the first object is a parent, in a school and he must have at least one child? You can enforce the relationship by writing a function and in the constructor, but then you also need to change the serialization/deserialization from json to enforce the rules, and of course you will write more code and you will forget to check it once, and there will be a bug.

And the most common problem of our times. The json is like:

{
    "id": 11,
    "name": "Maria",
    "age": 95,
    "children": [{
                "id": 5,
                "name": "Elena",
                "age": 28,
                "children": [{
                    "id": 6,
                    "name": "Dan",
                    "age": 60,
                    "children": [{
                        "id": 7,
                        "name": "Alex",
                        "age": 5
                    }],
                    {
                        "id": 9,
                        "name": "Alina",
                        "age": 32,
                        "children": [{
                            "id": 121,
                            "name": "Luiza",
                            "age": 0
                        }]
                    }
                }],

                {
                    "id": 23,
                    "name": "Petru",
                    "age": 70,
                    "children": [{
                            "id": 4,
                            "name": "Adrian",
                            "children": [{
                                "id": 45,
                                "name": "Denis",
                                "age": 12
                            }],
                        ]
                    }]
            }]
}

You have a single error but where? (Maria / Petru / Adrian - missing "age"). It is not only hard to validate it but it is hard to show explicitly where the error occurred.

Example 5: Functions

Let's try to find a string in another string. A function would be like:

int indexOf(string search, string what) ...

Which tells you that you will get an int, and you can pass two strings. First what if the strings are null? What if both strings are empty "", "". What if the result for "ab", "b" is 1248764 or -12. According to the function definition it is an int, and should be valid.

Example 6. Unit testing

To ensure the function above is well specified, we also use unit testing. Problem 1: unit testing doesn't care if it is static or dynamic typing. Problem 2 is very unit testing specific: Having enough tests, maintaining them when the function changes (like adding a new parameter), not enough testing, or too optimistic testing.


The solution proposed by Clojure.spec will be shown in part 2. And it is pretty cool! :)

Clojurescript/Reagent: How to start in 1 minute

What is a great development environment for web applications?

One that:

1.  can be started and configured extremely easy
2.  allows instant feedback
3.  can be done in a great programming language  
4.  doesn't need expensive tools

So Clojurescript. For clojure/clojurescript projects you need one tool installed: Leiningen, which will be used from the command line. (install from here. You need java installed before)

1. can be started and configured extremely easy

Create a new project

lein new figwheel hello_world -- --reagent

And start it:

cd hello_world

lein figwheeel

Now, go in your browser to localhost:3449

It is already there!!!

2. instant feedback

Open the code using some tool, preferably LightTable and do this change in src/hello_world/core.cljs in the hello_world component:

{:style {:color "red"}}

Save the file.



It is already in the browser!!

now, let's start that in console.

(in-ns 'hello_world.core)

Now let's change the text to Hello Dan!

(swap! app-state assoc :text "Hello Dan!")

Boom!



 It is already in the browser. Figwheel takes care of all that!


Now 3 and 4 are answered by clojurescript which is based on the great programming language clojure. We're also using Reagent (clojurescript library on top of Facebook React) and figwheel which allows all the instant feedback stuff.

Fighting complexity through functional composition, part 1: how to implement functional composition

The javascript code online:
http://jsfiddle.net/danbunea1/gz87dt5a/

Complexity

The absolute enemy in software (and other things as well) is complexity. Considering complexity as the opposite of simple, it makes our systems hard to understand, hard to debug and hard to extend or adapt. There is one very good talk of the great Rich Hickey, about this called: Simplicity Matters. The video: https://www.youtube.com/watch?v=rI8tNMsozo0. So let;s see how we can fight complexity in a practical example by using functional composition

The problem 

Considering that these days most of the integration is done through web services and JSON, we'll try to illustrate the complexity problem using an example from this area.

Requirement: in the json, we need to have a key “measurement”, that is mandatory, cannot be null, needs to be a string and cannot be empty string.

We’ll do a little bit of TDD here, starting with a test:

Python:

 class TestValidations(unittest.TestCase):  
   
   def validate_pair(self,json, is_valid, number_of_errors):  
     errors = validate_simplest_json(json)  
     print "",is_valid, errors, json  
     self.assertEquals(len(errors)==0,is_valid)  
     self.assertEquals(len(errors),number_of_errors)  
   
   def test_json_validation(self):  
     self.validate_pair({},False,1)  
   
   
 def validate_simplest_json(json):  
   errors = []  
   return errors  

All fail, which is great. Now let’s write the code, to check if the key is there:

 def validate_simplest_json(json):  
   errors = []  
   if not json.has_key("measurement"):  
     errors.append("measurement cannot be missing”)  
   return errors    

Pass. Now what about null? The test extends:

   def test_json_validation(self):  
     self.validate_pair({},False,1)  
     self.validate_pair({"measurement":None},False,1)  

the code to pass:

 def validate_simplest_json(json):  
   errors = []  
   if not json.has_key("measurement"):  
     errors.append("measurement cannot be missing")  
   else:  
     if json["measurement"]==None:  
       errors.append("measurement cannot be null”)  
   return errors  

Pass. Now let’s check if it is a string (or unicode):

   def test_json_validation(self):  
     self.validate_pair({},False,1)  
     self.validate_pair({"measurement":None},False,1)  
     self.validate_pair({"measurement":-1},False,1)  
     self.validate_pair({"measurement":{}},False,1)  
     self.validate_pair({"measurement":False},False,1)  
     self.validate_pair({"measurement":"abc"},True,0)  
     self.validate_pair({"measurement":u"Citroën"},True,0)  
   
 def validate_simplest_json(json):  
   errors = []  
   if not json.has_key("measurement"):  
     errors.append("measurement cannot be missing")  
   else:  
     if json["measurement"]==None:  
       errors.append("measurement cannot be null")  
     else:  
       if not isinstance(json["measurement"], str) and not isinstance(json["measurement"], unicode):  
         errors.append("measurement needs to string or unicode")  
   return errors  

Now we also need to check if it is not emty string:

   def test_json_validation(self):  
     self.validate_pair({},False,1)  
     self.validate_pair({"measurement":None},False,1)  
     self.validate_pair({"measurement":-1},False,1)  
     self.validate_pair({"measurement":{}},False,1)  
     self.validate_pair({"measurement":False},False,1)  
     self.validate_pair({"measurement":"abc"},True,0)  
     self.validate_pair({"measurement":u"Citroën"},True,0)  
     self.validate_pair({"measurement":""},False,1)  
   
 def validate_simplest_json(json):  
   errors = []  
   if not json.has_key("measurement"):  
     errors.append("measurement cannot be missing")  
   else:  
     if json["measurement"]==None:  
       errors.append("measurement cannot be null")  
     else:  
       if not isinstance(json["measurement"], str) and not isinstance(json["measurement"], unicode):  
         errors.append("measurement needs to string or unicode")  
       else:  
         if len(json["measurement"].strip())==0:  
           errors.append("measurement cannot be an empty string")  
   return errors  

All of the sudden, we hear we also need to make sure the length of the string is between 3 and 8 characters, and cannot be some reserved words like “password” or “archived"

Ok, so let’s code, expanding our tests:

   def test_json_validation(self):  
     self.validate_pair({},False,1)  
     self.validate_pair({"measurement":None},False,1)  
     self.validate_pair({"measurement":-1},False,1)  
     self.validate_pair({"measurement":{}},False,1)  
     self.validate_pair({"measurement":False},False,1)  
     self.validate_pair({"measurement":"abc"},True,0)  
     self.validate_pair({"measurement":u"Citroën"},True,0)  
     self.validate_pair({"measurement":""},False,1)  
     self.validate_pair({"measurement":"a"},False,1)  
     self.validate_pair({"measurement":"abcdefghijklmnefghij"},False,1)  
     self.validate_pair({"measurement":"password"},False,1)  
     self.validate_pair({"measurement":"archived"},False,1)  
     self.validate_pair({"measurement":"arCHived"},False,1)  

Then gradually we start coding the validation, arriving to:

 def validate_simplest_json(json):  
   errors = []  
   if not json.has_key("measurement"):  
     errors.append("measurement cannot be missing")  
   else:  
     if json["measurement"]==None:  
       errors.append("measurement cannot be null")  
     else:  
       if not isinstance(json["measurement"], str) and not isinstance(json["measurement"], unicode):  
         errors.append("measurement needs to string or unicode")  
       else:  
         lenm=len(json["measurement"].strip())  
         if lenm==0:  
           errors.append("measurement cannot be an empty string")  
         else:  
           if lenm<3: data-blogger-escaped-div="">  
             errors.append("measurement needs at least 3 characters")  
           elif lenm&gt;10:  
             errors.append("measurement needs at most 10 characters")  
           elif json["measurement"].strip().lower() in ["archived","password"]:  
             errors.append("measurement has a value which is not allowed")  
   return errors  

As requirements are added complexity grows. Now of course this code could be refactored, but eliminating the essential problem of complexity is very hard. Just imagine what will happen if at version 1.2 the customer will change the API and only allow the values to be a measurement unit like “0.12mm” or “13.2mg”. It will grow again and become more complex. Not pretty!

And having json with only one key is kind of rare… Usually the number of keys is a lot higher and of course the code a lot bigger. Bigger and more complex = disaster. In terms of code quality it will fail at being able to extend it easily and it will fail at being able to debug it easily.

The solution: implementing functional composition

Removing complexity can mean, more linear code, so let’s refactor it to be more linear:

 def validate_simplest_json_imperative_linear(json):  
   errors = []  
   should_exit=False  
   key = "measurement"  
   if not key_exists(json,key):  
    errors.append("{0} cannot be missing".format(key))  
    should_exit=True  
   
   if not should_exit:  
     if value_null(json, key):  
       errors.append("{0} cannot be null".format(key))  
       should_exit=True  
   
   if not should_exit:  
     if not is_string_or_unicode(json, key):  
       errors.append("{0} needs to string or unicode".format(key))  
       should_exit=True  
   
   if not should_exit:  
     if is_empty_string(json, key):  
       errors.append("{0} cannot be an empty string".format(key))  
       should_exit=True  
   
   
   if not should_exit:  
     lenm=len(json[key].strip())  
     if lenm<3: data-blogger-escaped-div="">  
       errors.append("{0} needs at least 3 characters".format(key))  
       should_exit=True  
     elif lenm&gt;10:  
       errors.append("{0} needs at most 10 characters".format(key))  
       should_exit=True  
   
   if not should_exit:  
     if json[key].strip().lower() in ["archived","password"]:  
       errors.append("{0} has a value which is not allowed".format(key))  
       should_exit=True  
   
   return errors  

And yes, all the tests still pass. But we’re far from over, although we do see a pattern by which each method is executed after the other… Hmm, now I’ll move all variables like json, key, errors and exit into a single object (a tuple) so that we don’t pass 4 parameters back and forth:

 ValidationState = namedtuple("ValidationState","json key errors exit”)  

then I will extract the actual validations in simple functions, like:

 def validate_simplest_json_imperative_linear_with_state(json):  
   initial_state = ValidationState(json=json, key="measurement",errors=[], exit=False)  
   
   state = validate_key_exists(initial_state)  
   
   if not state.exit:  
     state = validate_not_null(state)  
   
   if not state.exit:  
     state = validate_string_or_unicode(state)  
   
   if not state.exit:  
     state = validate_not_empty_string(state)  
   
   if not state.exit:  
     state = validate_length(state, 3,10)  
   
   if not state.exit:  
     state = validate_not_in(state, ["archived","password"])  
   
   return state.errors  

And the functions:

 def validate_key_exists(state):  
   print validate_key_exists.__name__,state  
   if not key_exists(state.json,state.key):  
     return state._replace(errors = state.errors+["{0} cannot be missing".format(state.key)])._replace(exit=True)  
   return state  
   
 def validate_not_null(state):  
   print validate_not_null.__name__,state  
   if value_null(state.json,state.key):  
     return state._replace(errors = state.errors+["{0} cannot be null".format(state.key)])._replace(exit=True)  
   return state  
   
 def validate_string_or_unicode(state):  
   print validate_string_or_unicode.__name__,state  
   if not is_string_or_unicode(state.json,state.key):  
     return state._replace(errors = state.errors+["{0} needs to string or unicode".format(state.key)])._replace(exit=True)  
   return state  
   
 def validate_not_empty_string(state):  
   print validate_not_empty_string.__name__,state  
   if is_empty_string(state.json,state.key):  
     return state._replace(errors = state.errors+["{0} cannot be an empty string".format(state.key)])._replace(exit=True)  
   return state  
   
 def validate_length(state, min, max):  
   print validate_length.__name__,state  
   lenm=len(state.json[state.key].strip())  
   if lenm  
     return state._replace(errors = state.errors+["{0} needs at least 3 characters".format(state.key)])._replace(exit=True)  
   elif lenm>max:  
     return state._replace(errors = state.errors+["{0} needs at most 10 characters".format(state.key)])._replace(exit=True)  
   return state  
   
 def validate_not_in(state,vals):  
   print validate_not_in.__name__,state  
   if state.json[state.key].strip().lower() in vals:  
     return state._replace(errors = state.errors+["{0} has a value which is not allowed".format(state.key)])._replace(exit=True)  
   return state  
   

The code looks is now a series of functions that run with the result of the previous function if the exit parameter is not set to True. So basically having 2 functions f,g they’ll be composed like:

initial_state = …

state = f(initial_state)

if not state.exit:

    return g(state)

And putting this in a function:

 def compose2(f, g):  
   def run(x):  
     result_f = f(x)  
     if not result_f.exit:  
       return g(result_f)  
     else:  
       return result_f  
   return run  

Using this we can now compose 2 functions into one:

 def validate_simplest_json_imperative_linear_with_state(json):  
   initial_state = ValidationState(json=json, key="measurement",errors=[], exit=False)  
   
   # state = validate_key_exists(initial_state)  
   #  
   # if not state.exit:  
   #   state = validate_not_null(state)  
   
   composed_function = compose2(validate_key_exists, validate_not_null)  
   state = composed_function(initial_state)  

But we don’t have only 2 function, we have more, so we write a reduce:

 #compose n functions  
 def compose(*functions):  
   return reduce(compose2, functions)  

and out function becomes:

 def validate_simplest_json_imperative_linear_with_state(json):  
   initial_state = ValidationState(json=json, key="measurement",errors=[], exit=False)  
   
   composed_function = compose(validate_key_exists, validate_not_null,validate_string_or_unicode,validate_not_empty_string)  
   state = composed_function(initial_state)  
   
   if not state.exit:  
     state = validate_length(state, 3,10)  
   
   if not state.exit:  
     state = validate_not_in(state, ["archived","password"])  
   
   return state.errors  

but we just hit a problem. He have some functions that have more parameters and we need to pass them. We’ll use closures:

 def create_validate_length(min, max):  
   def validate_length(state):  
     print validate_length.__name__,state  
     lenm=len(state.json[state.key].strip())  
     if lenm  
       return state._replace(errors = state.errors+["{0} needs at least 3 characters".format(state.key)])._replace(exit=True)  
     elif lenm>max:  
       return state._replace(errors = state.errors+["{0} needs at most 10 characters".format(state.key)])._replace(exit=True)  
     return state  
   return validate_length  
   
 def create_validate_not_in(vals):  
   def validate_not_in(state):  
     print validate_not_in.__name__,state  
     if state.json[state.key].strip().lower() in vals:  
       return state._replace(errors = state.errors+["{0} has a value which is not allowed".format(state.key)])._replace(exit=True)  
     return state  
   return validate_not_in  

And now the final validation code:

 def validate_simplest_functional_composition(json):  
   initial_state = ValidationState(json=json, key="measurement",errors=[], exit=False)  
   
   composed_function = compose(validate_key_exists, validate_not_null,validate_string_or_unicode,validate_not_empty_string, create_validate_length(3, 10), create_validate_not_in(["archived","password"]))  
   final_state = composed_function(initial_state)  
   
   return final_state.errors  

It is much better. It basically says: having an initial start of the system, run all these functions (validators) and at the end get a final state.

Now the entire code is available online and can also be run at:

http://runnable.com/VNMhoTKLSn9Tm0GI/fighting-complexity-through-functional-composition-for-python

a preview:

Or in Javascript: http://jsfiddle.net/danbunea1/gz87dt5a/

Conclusion: Why is this better?

Now you would think, how can a solution with ... lines of code be better then one with just 21. In part 2 of the article, called "Why is functional composition better" I will illustrate why, and how functional composition makes our code simpler, easier to understand, debug and change.