Python Notebooks in Production: How marimo Solves Jupyter’s Biggest Problems for Software Engineers

Notebooks are hard to maintain

Notebooks are helpful when checking logic and validating data. But they are not designed with software engineering best practices and are hard to maintain.

If you feel that notebooks:

Do the opposite of what a good tool should: they do not make it hard enough do the wrong thing (copypaste code, no version control, directly attaching prod resources to personal accounts)

Promote non-reusable code, bad habits, and are a security nightmare (due to the ipynb containing code and data)

Make it extremely hard to debug or test

This post is for you. Imagine being able to *explore data as you develop your production code. No need for complex workflows, copy-pasting code between notebooks and Python files, no complex bundling of notebooks, etc.

By the end of this post, you will know how to leverage the interactive & exploratory capabilities of Jupyter notebooks, along with SWE best practices, using marimo.

prerequistes

Create a repo and install the necessary libraries as shown below.

uv init marimo_notebooks 
cd marimo_notebooks
uv add jupyterlab marimo pandas polars pytest requests sqlalchemy

Note We will use uv run .. for the commands in the images, this will use the uv environment to run the commands.

Expand for code to follow along

In your repo, create a Python file.

touch marimo_example.py

Copy-paste this into the Python file.

import marimo

__generated_with = "0.19.4"
app = marimo.App()


@app.cell
def _():
    a = 10
    return (a,)


@app.cell
def _(a):
    a + 20
    return


@app.cell
def _(a):
    b = a + 30
    return (b,)


@app.function
def some_marimo_nb_function(a: int, b: int, c: int) -> int:
    return a + b + c


@app.cell
def _(b):
    b
    return


@app.cell
def _():
    b = 10
    return (b,)


@app.cell
def _():
    def non_importable_marimo_nb_function(a: int, b: int, c: int) -> int:
        return a + b + c

    print("""having a function and some executable code makes a function
    non importable""")
    return


@app.cell
def _(a):
    some_marimo_nb_function(a, 1, 2)
    return


@app.function
def test_some_marimo_nb_function():
    print("Running Tests...")
    assert some_marimo_nb_function(1, 2, 3) == 6


@app.cell
def _():
    print("standard python run...")
    return


@app.cell
def _(g):
    for j in range(10):
        if j == 5:
            print(g)
            print(j)
    return


@app.cell
def _():
    for i in range(10):
        if i == 8:
            breakpoint()
            print(i)
    return


@app.cell
def _():
    import sqlalchemy

    # Create an in-memory SQLite database with SQLAlchemy
    sqlite_engine = sqlalchemy.create_engine("sqlite:///:memory:")
    return (sqlite_engine,)


@app.cell
def _():
    import marimo as mo

    return (mo,)


@app.cell
def _(mo, sqlite_engine):
    _df = mo.sql(
        """
        create TABLE employee (id int, name VARCHAR)
        """,
        engine=sqlite_engine,
    )
    return


@app.cell
def _(mo, sqlite_engine):
    _df = mo.sql(
        """
        INSERT into employee values (1, "john"), (2, "doe")
        """,
        engine=sqlite_engine,
    )
    return


@app.cell
def _(mo, sqlite_engine):
    _df = mo.sql(
        """
        SELECT * FROM employee
        """,
        engine=sqlite_engine,
    )
    return


@app.cell
def _():
    import polars as pl
    import requests

    url = "https://raw.githubusercontent.com/vega/vega-datasets/master/data/cars.json"
    response = requests.get(url)
    df = pl.read_json(response.content)

    return (df,)


@app.cell
def _(df, mo):
    transformed_df = mo.ui.dataframe(df)
    transformed_df
    return (transformed_df,)


@app.cell
def _(transformed_df):
    transformed_df
    return


@app.cell
def _():
    return


if __name__ == "__main__":
    app.run()

Run marimo on this file.

uv run marimo edit marimo_example.py

Combine Software Engineering Best Practices with Exploratory Pipeline Development

By using marimo, you can get the best of both worlds:

Exploratory data pipeline development, where you can see the shape of your data as you develop your pipeline
Code saved as Python files enabling: tests, debugging, importable functions, version control, git diffs, etc

Version Control

Jupyter notebooks are json files containing both the code and any output. It is extremely hard for a developer to identify code changes.

Marimo saves its notebooks as Python files. Each marimo cell block is a function, and it does not save its output, making it easy to identify code changes.

Git diffs are simple.

Code Reusability

To productionize code in a Jupyter notebook, you will need to copy and paste it into a Python file. Manual copy-paste can lead to missed logic, incorrect ordering, and other issues.

With marimo, you can create functions in your notebook and use them in other Python scripts.

This is possible since Marimo saves notebooks as Python files.

Write Tests as You Would for a Python File

As marimo notebooks are just Python files, you can write tests for them. In addition, you can also add tests directly in the notebook and run them with pytest.

Debugging Code in Marimo Notebook

You can set a debug point in your notebook using breakpoint(). When the logic gets to this point, a debug session is started. You can inspect variables and debug your code.

You can also start a debugger on exception. This is particularly helpful when you need to check the state of your code after an error.

Auto Format Notebooks

Marimo has a built-in notebook format checker with the check command. You can fix format issues with the —fix flag.

Data Manipulations: SQL, and Visual No-Code Transformations

Establish connections to the DB and query them directly with SQL.

You can use it to start a no-code transformation on dataframes. You can also access the corresponding transformation code.

Use the UI to transform data without code.

Click on the </> Python Code tab to see its corresponding transformation code.

Reactivity & Dependency Tracking

When a variable changes, its dependent variables will change as well (aka reactivity). Marimo tracks the variable’s dependence across the entire notebook.

You can see the dependency graph for any variable using the dependency tab.

Note A variable can be initialized in only one cell, as it makes maintenance much simpler.

Conclusion

To recap, we saw how marimo enables you to write notebooks that are

Tired of using Jupyter and having to manually check the order of code execution, tired of having to copy and paste code between notebooks and IDEs?

Give marimo a try. You can quickly convert your jupyter notebooks to marimo with marimo convert your_notebook.ipynb -o your_notebook.py command and get started today!