subprocessing

Subprocess is a way to run executable files and binaries in python (including other python files). It is an incredibly useful tool for setup scripts, project build scripts, and other complicated tasks.

Definitions

Some terms to help you navigate the post

Subprocess

A python module that allows you to run subprocesses to complete tasks such as running executable files, binaries, and even other python files.

A basic example would be to run another python file from a python file:

import subprocess

# Instantiates another python interpreter inside the current session and runs other_python_file.py
subprocess.run(["python", "other_python_file.py"])

You could also use it to run files in other languages like Go. Here is an example go file to run:

helloWorld.go

import "fmt"

func main(){
    fmt.Print("Hello World!")
}

This file can then be run in python using:

import subprocess

# Equivalent to running go run helloWorld.go
subprocess.run(["go", "run", "helloWorld.go"])

Files that are in other directories can be run by pathing the final argument to the correct file, for example:

import subprocess

# Equivalent to running go run helloWorld.go, and assumes it is 3 directories deep
subprocess.run(["go", "run", "/path/to/file/helloWorld.go"])

Usage

In this repo there are two demos you can run;

Subprocess Demo

You can run the basic demo by python subprocess_demo.py or python3 subprocess_demo.py

C Demo

1. Make sure you have gcc installed so you can compile the C code
2. First cd into the /c_demo folder.
3. run python cross_language_demo.py or python3 cross_language_demo.py

Real World Applications

Python subprocesses are incredibly useful for writing scripts that need to interface with non-python executable files and/or binaries. For example if you have a project that is cross-language (let's say a backend in Go and Rust, with a frontend that needs a webpack build), you can run all the necessary migrations, dependency fetching, compiling, and running from a single python script.

Additional Information

Reference for subprocessing module

Files

testScriptBatchDemo.bat

@echo off
REM File runs an echo with some text and a dir. It gets run from
REM subprocess_demo.py if the machine running it is windows
echo "This is test_script.bat running"
dir

testScriptShellDemo.sh

# File runs an echo and an ls and is called from
# subprocess_demo.py if the machine running it is unix/mac
echo "This is test_script.sh running"
ls

subprocessing_demo.py

""" Subprocess is a module that lets you run commands and executable files
from a python script."""

import os # Used to determine OS and run cls/clear commands
import subprocess # Used to run external commands/executable files

# Clears terminal after starting file: this helps with legibility
if os.name == "nt":# PORT: If on windows then run the cls command
    os.system("cls")
else: # PORT: If on unix/mac then run the clear comand
    os.system("clear")
print("This is running in the subprocess_demo.py file\n")

# Run the test script for respective OS's
if os.name == "nt": # PORT: If on windows then run the .bat version
    print("Starting up test_script.bat\n\n")
    subprocess.run(["testScriptBatchDemo.bat"])
else: # PORT: If on unix/mac then run the .sh version
    print("Starting up test_script.sh\n\n")
    subprocess.run(["testScriptShellhDemo.sh"])

/c_demo/cross_language_demo.py_demo.py

"""This file is used to show off a real world example of creating a build
script in python and using it to build+run a C program.
NOTE: This demo assumes you have gcc installed"""

import os # Used to determine if machine is windows or unix/macOS
import subprocess # Used to run commands from python

def compile(filename, binary_filename):
    """Function to compile the provided file in gcc"""
    # Below is equivalent to running: gcc -o hello_world hello_world.c
    print(f"Creating binary: {binary_filename} From source file: {filename}\n")
    subprocess.run(["gcc", "-o", binary_filename, filename])

def run_binary(binary_filename):
    """Runs the provided binary"""
    print(f"Running binary: {binary_filename}\n")
    subprocess.run([binary_filename])
if __name__ == "__main__":
    compile("hello_world.c", "hello_world")
    if os.name =="nt": # If on windows
        run_binary("hello_world.exe")
    else: # If on unix/mac
        run_binary("hello_world")
        print("Binary run")

/c_demo/hello_world_demo.c

#include<stdio.h>
// This file is just to show off running a binary;
// i.e. a .exe in windows and regular binary in unix/macOS
int main(int argc, char* argv[]){
    // Prints 'Hello, World!' then exits
    printf("Hello, World!\n");
    return 0;
}