How does “Python -m” Work?

In yesterday’s post, we discussed various command line flags one can run Python scripts with and their purpose.

Here’s the visual from that post for a quick recap:

In that post, we didn’t entirely discuss the python -m as it required additional background details.

So, let’s continue the discussion on that today.

What does python -m do?

Simply put, this command line flag lets us run a module as a script.

• A script is a Python file we would execute — python code.py.

• A module is a Python file we would import code from — import code.

So, in a way, the -m option lets us run the module as if it were a script, as shown below:

python -m code

Now the question is: “Why would we do that?”

Let’s understand this today and also cover some background details.

Background

In Python, the statements under the main block (if __name__ == "__main__") only run when the Python file is executed as a script.

For instance, running the utils.py file below as a script executes the print_hello() statement specified under the main block:

However, if we were to import utils in another file, i.e., use the utils file as a module, the contents under its main block will not be executed during import.

This can be verified from the demonstration below:

In the above code:

• We import utils in code.py.

• Executing code.py does not produce any output, proving that the main block runs only when the file is executed as a script.

That said, another way to execute the utils.py file is using the -m flag:

As depicted above, we can also use the -m flag instead of python utils.py.

But as “executing the script” and the “-m flag” produce the same output, why and when would we prefer the ‘-m flag’ route?

In my experience, there are two profound usages of the -m command line flag.

Usage #1: Expose a command for a library

At times, the developers of a library want to expose a command as part of the library’s utility (one that can be executed in the command line).

Here, following the natural way of invoking the script is tedious because libraries are not installed in the working directory.

Instead, they are typically available in the site-packages directory.

But invoking the desired file as a script from the working directory will only work when the user provides the full absolute path, as depicted below:

Another way to use the library’s functionality (which you already know) is when we import it in a script:

But that is not the objective here as we intend to expose a command as part of the library’s utility — one that can be executed in the command line.

So, in a way, we are only left with the option of specifying the absolute path, which is tedious and long.

The -m flag resolves this issue, as demonstrated below:

It provides the flexibility of using the file as a module but running it as a script, which is evident from the image above.

The above command does not require us to specify the absolute path because we are using it as a module, not as a script. Thus, it works absolutely fine, as we also saw this when we imported the timeit library above.

This way, the developers can expose a command for the library for better utility.

So effectively, the -m coveys the following message to the Python interpreter.

Usage #2: Ensure library installation in the desired environment

Typically, a system may have many different versions of Python and pip (the Python package manager).

Due to many different pip versions, it can be challenging to manage and ensure that we are installing a library in the desired Python version.

The pip library uses the -m flag to expose the following command, which ensures that we install the library in the correct Python version:

python -m pip install <library-name>

In fact, that is the reason why library installation pages typically provide the above command instead of pip install <library-name>.

This ensures that we install the library in that specific version which is accessed using the ‘Python’ command.

Also, say we have both Python 3.9 and Python 3.10 installed in our system.

We can install a library in Python 3.9 as follows:

python3.9 -m pip install <library-name>

Or in Python 3.10 as follows:

python3.10 -m pip install <library-name>

Isn’t that a cool and handy Python command line flag?

👉 Over to you: Do you know any other usages of the -m flag? If yes, let me know :)

Are you overwhelmed with the amount of information in ML/DS?

Every week, I publish no-fluff deep dives on topics that truly matter to your skills for ML/DS roles.

For instance:

• Quantization: Optimize ML Models to Run Them on Tiny Hardware

• A Crash Course on Causality – Part 1

• A Crash Course on Causality – Part 2

• A Beginner-friendly Introduction to Kolmogorov Arnold Networks (KANs)

• Implementing KANs From Scratch Using PyTorch

• A Practical Guide to Scaling ML Model Training

• 5 Must-Know Ways to Test ML Models in Production (Implementation Included)

• A Beginner-Friendly Guide to Multi-GPU Model Training

• Understanding LoRA-derived Techniques for Optimal LLM Fine-tuning

• 8 Fatal (Yet Non-obvious) Pitfalls and Cautionary Measures in Data Science

• Implementing Parallelized CUDA Programs From Scratch Using CUDA Programming

• You Are Probably Building Inconsistent Classification Models Without Even Realizing

• And many many more.

Join below to unlock all full articles:

SPONSOR US

Get your product in front of 82,000 data scientists and other tech professionals.

Our newsletter puts your products and services directly in front of an audience that matters — thousands of leaders, senior data scientists, machine learning engineers, data analysts, etc., who have influence over significant tech decisions and big purchases.

To ensure your product reaches this influential audience, reserve your space here or reply to this email to ensure your product reaches this influential audience.