{"id":912,"date":"2019-09-08T11:49:59","date_gmt":"2019-09-08T09:49:59","guid":{"rendered":"http:\/\/35.180.88.53\/?p=912"},"modified":"2019-10-28T12:50:25","modified_gmt":"2019-10-28T11:50:25","slug":"best-practices-for-using-functional-programming-in-python","status":"publish","type":"post","link":"https:\/\/www.sergilehkyi.com\/uk\/2019\/09\/best-practices-for-using-functional-programming-in-python\/","title":{"rendered":"Best Practices for Using Functional Programming in Python"},"content":{"rendered":"\n

Introduction<\/h2>\n\n\n\n

Python is a very versatile, high-level programming language. It has a generous standard library, support for multiple programming paradigms, and a lot of internal transparency. If you choose, you can peek into lower layers of Python and modify them \u2013 and even modify the runtime on the fly as the program executes.<\/p>\n\n\n\n

I\u2019ve recently noticed an evolution in the way Python programmers use the language as they gain more experience. Like many new Python programmers, I appreciated the simplicity and user friendliness of the the basic looping, function, and class definition syntax when I was first learning. As I mastered basic syntax, I became curious about intermediate and advanced features like inheritance, generators, and metaprogramming. However, I wasn\u2019t quite sure when to use them, and would often jump at opportunities to practice that weren\u2019t a great fit. For a while, my code became more complex and harder to read. Then, as I kept iterating \u2013 especially if I kept working on the same codebase \u2013 I gradually reverted back to mostly using functions, loops, and singleton classes.<\/p>\n\n\n\n

With that being said, the other features exist for a reason, and they\u2019re important tools to understand. \u201cHow to write good code\u201d is obviously an expansive topic \u2013 and there\u2019s no single right answer! Instead, my goal with this blog post is to zero in on a specific aspect: functional programming as applied to Python. I\u2019ll dig into what it is, how it can be used in Python, and how \u2013 according to my experience \u2013 it\u2019s used best.<\/p>\n\n\n\n

<\/strong>What is functional programming?<\/h2>\n\n\n\n

Functional programming, or FP, is a coding paradigm in which the building blocks are immutable values and \u201cpure functions\u201d that share no state with other functions. Every time a pure function has a given input, it will return the same output \u2013 without mutating data or causing side effects. In this sense, pure functions are often compared to mathematical operations. For example, 3 plus 4 will always equal 7, regardless of what other mathematical operations are being done, or how many times you\u2019ve added things together before.<\/p>\n\n\n\n

With the building blocks of pure functions and immutable values, programmers can create logical structures. Iteration can be replaced with recursion, because it is the functional way to cause the same action to occur multiple times. The function calls itself, with new inputs, until the parameters meet a termination condition. In addition, there are higher-order functions, which take in other functions as input and\/or return them as output. I\u2019ll describe some of these later on.<\/p>\n\n\n\n

Although functional programming has existed since the 1950s, and is implemented by a long lineage of languages, it doesn\u2019t fully describe a programming language. Clojure, Common Lisp, Haskell, and OCaml are all functional-first languages with different stances on other programming language concepts, like the type system and strict or lazy evaluation. Most of them also support side effects such as writing to and reading from files in some way or another \u2013 usually all very carefully marked as impure.<\/p>\n\n\n\n

<\/strong>What does Python support?<\/h2>\n\n\n\n

Though Python is not primarily a functional language, it is able to support functional programming relatively easily because everything in Python is an object. That means that function definitions can be assigned to variables and passed around.<\/p>\n\n\n\n

def add(a, b):\n    return a + b\n\nplus = add\n\nplus(3, 4)  # returns 7<\/code><\/pre>\n\n\n\n
Lambda<\/h3>\n\n\n\n
The \u201clambda\u201d syntax allows you to create function definitions in a declarative way. The keyword lambda comes from the greek letter used in the formal mathematical logic for describing functions and variable bindings abstractly, \u201clambda calculus\u201d<\/a>, which has existed for even longer than functional programming. The other term for this concept is \u201canonymous function\u201d, since lambda functions can be used in-line without ever actually needing a name. If you do choose to assign an anonymous function to a variable, they perform exactly the same as any other function.<\/p>\n\n\n\n
(lambda a, b: a + b)(3, 4)  # returns 7\n\naddition = lambda a, b: a + b\naddition(3, 4)  # returns 7<\/code><\/pre>\n\n\n\n
The most common place I see lambda functions \u201cin the wild\u201d is for functions that take in a callable. A \u201ccallable\u201d is anything that can be invoked with parentheses \u2013 practically speaking classes, functions and methods. Amongst those, the most common use is to declare a relative prioritization via the argument key when sorting data structures.<\/p>\n\n\n\n
authors = ['Octavia Butler', 'Isaac Asimov', 'Neal Stephenson', 'Margaret Atwood', 'Usula K Le Guin', 'Ray Bradbury']\nsorted(authors, key=len)  # Returns list ordered by length of author name\nsorted(authors, key=lambda name: name.split()[-1])  # Returns list ordered alphabetically by last name.<\/code><\/pre>\n\n\n\n
The downside to inline lambda functions is that they show up with no name in stack traces, which can make debugging more difficult.<\/p>\n\n\n\n
<\/strong>Functools<\/h3>\n\n\n\n
The higher-order functions that are the meat-and-potatoes of functional programming are available in Python either in builtins or via the functools library. map and reduce may ring a bell as a way to run distributed data analysis at scale, but they are also two of the most important higher-order functions. map applies a function to every item in a sequence, returning the resultant sequence, and reduce uses a function to collect every item in a sequence into a single value.<\/p>\n\n\n\n
val = [1, 2, 3, 4, 5, 6]\n\n# Multiply every item by two\nlist(map(lambda x: x * 2, val)) # [2, 4, 6, 8, 10, 12]\n# Take the factorial by multiplying the value so far to the next item\nreduce(lambda: x, y: x * y, val, 1) # 1 * 1 * 2 * 3 * 4 * 5 * 6<\/code><\/pre>\n\n\n\n
There are a pile of other higher order functions that manipulate functions in other ways, notably partial, which locks in some of the parameters to the function. This is also known as \u201ccurrying\u201d, a term named after FP pioneer Haskell Curry.<\/p>\n\n\n\n
def power(base, exp):\n     return base ** exp\ncube = partial(power, exp=3)\ncube(5)  # returns 125<\/code><\/pre>\n\n\n\n
For a detailed tour of introductory FP concepts in Python, written in the way a functional-first language would use them, I recommend Mary Rose Cook\u2019s article here<\/a>.<\/p>\n\n\n\n
<\/strong>Decorators<\/h3>\n\n\n\n
Higher-order functions are also baked into everyday Python via decorators. One way of declaring decorators reflects that, and the @ symbol is basically a syntactic sugar for passing in the decorated function as an argument to the decorator. Here is a simple decorator that sets up retries around a piece of code and returns the first successful value, or gives up and raises the most recent exception after 3 attempts.<\/p>\n\n\n\n
def retry(func):\n    def retried_function(*args, **kwargs):\n        exc = None\n        for _ in range(3):\n            try:\n               return func(*args, **kwargs)\n            except Exception as exc:\n               print(\"Exception raised while calling %s with args:%s, kwargs: %s. Retrying\" % (func, args, kwargs).\n\n        raise exc\n     return retried_function\n\n@retry\ndef do_something_risky():\n    ...\n\nretried_function = retry(do_something_risky)  # No need to use `@`<\/code><\/pre>\n\n\n\n
This decorator leaves the input and output types and values as exactly the same \u2014 but that\u2019s not a requirement. Decorators can add or remove arguments or change their type. They can also be configured via parameters themselves. I want to stress that decorators themselves are not necessarily \u201cpurely functional\u201d; they can (and often do, as in the example above) have side effects \u2013 they just happen to use higher order functions.<\/p>\n\n\n\n
Like many intermediate or advanced Python techniques, this is very powerful and often confusing. The name of the function you called will be different from the name in the stack traces, unless you use the functools.wraps decorator to annotate. I have seen decorators do very complicated or important things, like parse values out of json blobs or handle authentication. I\u2019ve also seen multiple layers of decorators on the same function or method definition, which requires knowing the decorator application order to understand. I think it can be helpful to use the builtin decorators like `staticmethod` or write simple, clearly named decorators that save a lot of boilerplate, but especially if you want to make your code compatible with type checking, anything that changes the input or output types can easily edge into \u201ctoo clever\u201d.<\/p>\n\n\n\n
<\/strong>My recommendations<\/h2>\n\n\n\n
Functional programming is interesting, and learning paradigms that are outside your current comfort zone is always good for building flexibility and allowing you to look at problems in different ways. However, I wouldn\u2019t recommend writing a lot of functional-first Python, especially in a shared or long-lived codebase. Aside from the pitfalls of each feature I mentioned above, here\u2019s why:<\/p>\n\n\n\n