The Most Under-appreciated Technique To Speed-up Python

Python’s default interpreter — CPython, isn’t smart.

It serves as a standard interpreter for Python and offers no built-in optimization.

Instead, use the Cython module.

Cython converts your Python code into C, which is fast and efficient.

Steps to use the Cython module:

• Load the Cython module (in a separate cell of the notebook): %load_ext Cython.

• Add the Cython magic command at the top of the cell: %%cython -a.

• When using functions, specify their parameter data type.

def func(int number): ...

• Define every variable using the “cdef” keyword and specify its data type.

cdef int a = 10

Once done, Cython will convert your Python code to C, as depicted below:

This will run at native machine code speed. Just invoke the method:

>>> foo_c(2)

The speedup is evident from the image below:

• Python code is slow.

• But Cython provides a 100x speedup.

Why does this work?

Essentially, Python is dynamic in nature.

For instance, you can define a variable of a specific type. But later, you can change it to some other type.

a = 10a = "hello" # Perfectly legal in Python

These dynamic manipulations come at the cost of run time. They also introduce memory overheads.

However, Cython lets you restrict Python’s dynamicity.

We avoid the above overheads by explicitly specifying the variable data type.

cdef int a = 10a = "hello" ## Raises error

The above declaration restricts the variable to a specific data type. This means the program would never have to worry about dynamic allocations.

This speeds up run-time and reduces memory overheads.

Isn’t that cool?

👉 If you liked this post, don’t forget to leave a like ❤️. It helps more people discover this newsletter on Substack and tells me that you appreciate reading these daily insights. The button is located towards the bottom of this email.

👉 Over to you: What are some other ways to speed up Python?

Thanks for reading!

Whenever you’re ready, here are a couple of more ways I can help you:

• Get the full experience of the Daily Dose of Data Science. Every week, receive two curiosity-driven deep dives that:

  • Make you fundamentally strong at data science and statistics.

  • Help you approach data science problems with intuition.

  • Teach you concepts that are highly overlooked or misinterpreted.

• Promote to over 28,000 subscribers by sponsoring this newsletter.

👉 Tell the world what makes this newsletter special for you by leaving a review here :)

👉 If you love reading this newsletter, feel free to share it with friends!