The Past, Present, and Future of Jovian

In late 2023, I decided to revamp Jovian, a learning platform for Data Science that I (and a small team) had been building and operating since 2019. My goal was to refresh the design, modernize the tech stack, reduce cloud computing costs, and improve the user experience.

I also wanted to remove some features. For instance, as part of our data science courses, users could create, store, and run Jupyter notebooks within Jovian itself. I felt this was no longer necessary, as Google Colab is a free alternative that works equally well, if not better.

Pruning the Codebase

As a first step, I deleted over 50,000 lines of code in 3 days from our frontend codebase, and continued to remove unused features for a couple of months. It led to a radical simplification of the codebase and the app's user experience, while reducing the codebase size by 75%.

Next, I started cleaning up API routes and database tables from our backend codebase, built with Python, Flask, and SQLAlchemy. While it took much more time and effort, the reduction here was even more drastic, as I removed over 90% of the API routes and 85% of the code:

I also removed most automated emails, turned off cron jobs, shut down our self-hosted BinderHub cluster, removed/optimized many database queries, and removed our landing page. Two years in, I can confidently claim nearly nothing of essence has been lost.

Optimizing Cloud Costs

Our Next.js frontend was hosted on Vercel ($20/mo), and our Flask backend was hosted on Google Kubernetes Engine (~$450/mo). After some investigation into self-hosting web applications, I migrated both of them to a single cloud virtual machine on Hetzner ($5/mo).

Our database was hosted on Supabase (~$250/mo) and had over 75 GB of data. After deleting unused tables, rows, and columns, I was able to reduce the database size to under 3 GB. I then migrated it over to the Hetzner VM hosting the frontend & backend servers.

We also had ~3 TB of user-uploaded Jupyter notebooks stored in Google Cloud Storage (~$160/mo). I removed the version control system we had in place and migrated the latest version of every Jupyter notebook to Cloudflare R2, costing $7.50/mo for 300 GB in storage.

I applied to Cloudflare's startup program and obtained credits to cover the R2 bill for a year. Overall, I've been able to cut Jovian's cloud bill from over $1000 per month to under $5 per month, while the site's traffic and our monthly revenue have remained largely stable.

Tech and Design Revamp

I'm currently working on a ground-up rewrite of Jovian using React 19, Next.js 16, Tailwind v4, Drizzle ORM, and Shadcn UI. Aside from revamping the tech stack, I'm also aiming to improve the user interface with clean, minimal, and modern design aesthetics.

Having all parts of the tech stack on a single cloud VM and reducing the number of database queries has improved the application's performance dramatically. I want to take this even further by removing the Python backend and replacing the Postges database with SQLite.

The Way Forward

Video lessons in our courses have associated Jupyter notebooks. Currently, these notebooks are hosted on Jovian itself, but I intend to move all lesson notebooks to GitHub. This will reduce our dependence on Cloudflare R2 and enable direct execution on Google Colab.

I'm working with a former student to gradually update our course videos and Jupyter notebooks. Instructions on how to run notebooks or submit assignments/projects are no longer valid and are being revised. Outdated code and broken links are also being fixed.

I have also disabled uploading/creation of new Jupyter notebooks on Jovian, encouraging learners to use Google Colab instead. I plan to delete all user-uploaded notebooks and completely remove Cloudflare R2 from the stack, achieving a true single-machine setup.

From Jovian to SwiftAce

What does the revamp of Jovian mean for SwiftAce (the customizable open-source platform I'm building to help educators self-host online courses)? Here's how I see it playing out:

1. The revamped version of Jovian will serve as a reference for the functionality, design, and user experience I want to achieve with SwiftAce, both for learners and educators.

2. We can strip out the Jovian-specific pieces of the app and make them configurable, while retaining all the other generic parts, to create an initial version of SwiftAce.

3. Using the initial version as a reference implementation, we can rebuild SwiftAce based on the design principles outlined earlier and try to achieve extreme extensibility.

This is going to take a while. I'd love to be able to do this sustainably over the long term, so there might be some detours along the way as I figure out how to make this happen.