Django & HTMX Server-Side Datatables

In this short guide, you will learn how to create a modern, fast UI for browsing large datasets with Django & HTMX. When dealing with large amounts of data, for example, thousands or millions of rows the DOM will be too slow. In cases like this, it is better to let the database do all the heavy lifting.

Prerequisites

Basic knowledge of Django
Basic knowledge of HTMX

LEVEL - 💻 💻 Beginner-Intermediate. Basic knowledge of Django and HTMX is assumed. The codebase for the tutorial can be found here . Clone the repository if you want to follow along.

Introduction

Our objective for this tutorial is to set up a view for browsing a large dataset i.e. 100,000 plus rows. We will be using the django-tables2 library, to simplify turning tabular data into HTML tables. We will use TailwindCSS to style the table view and HTMX for Ajax requests and DOM manipulation.

The Web application

We will be building a simple search metrics app that displays a table of search records.

Project Structure & Best Practices for maintainability

If you have already cloned the repo, this is a good time to go over the basic project structure, and briefly mention best practices for code quality and maintainability.

|django-htmx 
├── config                  --> app configuration
├── apps 
|   ├── search_metrics      --> main app and focus of this article
├── static
|   ├── js                  --> includes htmx scripts
|   ├── css                 --> site css
├── templates
|   ├── base.html           --> base template, includes htmx scripts
|   └── common
|      └── django_tables2.html
|   └── search_metrics
|      ├── htmx             --> keep htmx components in separate folders 
|         ├── table.html
|      ├── metrics.html
└── .pre-commit-config.yml  --> linting & formatting tools.
└── Makefile                --> shortcuts for convenience
└── requirements.txt        --> project dependencies
└── manage.py

Notes

apps - create a separate apps folder for all of your internal apps.
static - in the static folder you will notice we have included the htmx.min.js script instead of downloading from a CDN for privacy, security, and improved performance. See this article for more info.
search_metrics/htmx - keep your htmx templates in a separate folder from your main templates. This will be particularly useful when your app grows and there are a lot of small htmx views/partial templates.
.pre-commit-config.yml - includes pre-commit configuration for linting, formatting the code base with flake8, black, and isort. This is the most important framework for automating & managing the various code quality tools in the project.
Makefile - Use a Makefile to create shortcut commands for convenience. running python manage.py runserver 0.0.0.0:8000 gets tiring pretty quickly. make run is so much better and easier to remember.
requirements.txt - includes all of our project dependencies.

If you are ready you can follow the instructions in the README to set up a local environment. The rest of the article will focus on htmx and server-driven tables.

Setting up the backend

Database Table

search_metrics/models.py

from django.db import models
from model_utils.models import UUIDModel


class SearchMetric(UUIDModel):
    client_ip = models.GenericIPAddressField()
    hits = models.IntegerField(null=True)
    search_date = models.DateTimeField(auto_now_add=True)
    search_term = models.CharField(max_length=300, blank=True)

Create a standard Django model. We are using the django-model-utils library to use a UUID as the primary key.

Table Class

search_metrics/tables.py

import django_tables2 as tables

from apps.search_metrics.models import SearchMetric


class SearchMetricsTable(tables.Table):
    class Meta:
        model = SearchMetric

    @classmethod
    def render_paginated_table(cls, request):
        """Render paginated table"""
        table = cls(data=SearchMetric.objects.all())
        table.paginate(page=request.GET.get("page", 1), per_page=25)
        return table

Notes

The Table class will create a table from any iterable that supports len() and contains items that expose key-based access to column values. The library has first-class support for Django querysets. This means that when creating a new table class, you can specify a model in the class Meta and all the columns, field types will be inferred from the model.

We have included a method to render a paginated tabled based on request parameters. This is a slight departure from the examples given in the library where business logic is included in the view.

HTMX views

search_metrics/views.py

from django.core.management import call_command
from django.shortcuts import render
from django.views.decorators.http import require_http_methods, require_GET

from apps.search_metrics.models import SearchMetric
from apps.search_metrics.tables import SearchMetricsTable


@require_GET
def search_metrics_view(request):
    
    template_name = "search_metrics/metrics.html"
    if request.htmx:
        template_name = "search_metrics/htmx/table.html"
        
    context = {"table": SearchMetricsTable.render_paginated_table(request)}
    
    return render(request, template_name, context)

Notes

When using htmx, prefer simple, function-based views over Django's generic class-based views. Your code will much easier to read and reason about.

Wire up the urls

search_metrics/urls.py

from django.urls import re_path, path

from apps.search_metrics.views import(
 search_metrics_view, seed_db_view, clear_db_view
)

app_name = "search_metrics"
urlpatterns = [
    re_path(r"^(?:page-(?P<page_number>\d+)/)?$", search_metrics_view, name="search-metrics-table"),
]

Finally, wire up the views in the app's urls.py file. Don't forget to include the search_metrics app's URL patterns in the main application in config/urls.py.

Add Frontend Templates

Base table template

templates/common/django_tables2.html

.....
<li>
    <button hx-get="{% querystring table.prefixed_page_field=table.page.previous_page_number %}" 
            hx-target="#tableContainer" 
            hx-swap="outerHTML">
        &larr;
    </button>
</li>
....

Notes

The django_tables2 library provides a basic template for rendering an HTML table . We have modified that template to include some custom styling with using tailwindcss.

The pagination at the bottom of the pages has been enhanced with htmx

Anchor tags for the prev, next, and page buttons have been replaced with an hx-get attribute which allows us to make ajax requests directly from the html.
the hx-target attribute specifies which element should be swapped out with the response
the hx-swap attribute allows you to specify how the response will be swapped relative to the target element. In this case, the entire element with id tableContainer will be swapped with the response.

Page and table component templates

templates/search_metrics/metrics.html

{% extends "base.html" %}

{% block content %}
    <!--Add htmx headers hear, ca-->
    <div class="flex flex-col mx-auto container">
        <h2 class="font-semibold text-2xl mt-12 text-gray-700">Search Metrics</h2>
        <div class="mx-auto container mt-5">
            {% include "search_metrics/htmx/table.html" %}
        </div>
    </div>
{% endblock %}

templates/search_metrics/htmx/table.html

{% load render_table from django_tables2 %}

<!--Include id="tableContainer" in enclosing container -->
<div id="tableContainer" class="overflow-x-auto max-w-5xl">
    <p class="font-semibold text-xs px-2 text-gray-600">Showing page {{ table.page.number }} of {{ table.paginator.num_pages }}</p>
    {% render_table table "common/django_tables2.html" %}
</div>

Notes

{% include "search_metrics/htmx/table.html" %} - the table component below is included as its own separate template. We split up the files so that we can call the correct template from the view.

id="tableContainer" - we indicate the target element to be swapped out using the tableContainer id. The htmx library will find this element and swap it out with the response from the backend.

Are you building a SaaS product and want to launch like ASAP? Check out the Vanty Starter Kit. Our Django boilerplate has everything you need to launch your app today!

Conclusion

That's it. You have learned how to create a fast, modern UI for viewing large datasets in table format.

The code for the tutorial includes some other enhancements such as a management command for seeding the database and additional views for adding or removing data from the db.

In this project, we used sqlite3 as our database, this is ok for small databases or hobby projects. For 'more serious' projects use PostgreSQL.

Search This Blog

Web Development Notes