From patchwork Thu Mar 8 00:31:15 2012 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Michael-Doyle Hudson X-Patchwork-Id: 7152 Return-Path: X-Original-To: patchwork@peony.canonical.com Delivered-To: patchwork@peony.canonical.com Received: from fiordland.canonical.com (fiordland.canonical.com [91.189.94.145]) by peony.canonical.com (Postfix) with ESMTP id C398B23E67 for ; Thu, 8 Mar 2012 00:31:19 +0000 (UTC) Received: from mail-iy0-f180.google.com (mail-iy0-f180.google.com [209.85.210.180]) by fiordland.canonical.com (Postfix) with ESMTP id 563F5A18386 for ; Thu, 8 Mar 2012 00:31:19 +0000 (UTC) Received: by iage36 with SMTP id e36so12590015iag.11 for ; Wed, 07 Mar 2012 16:31:18 -0800 (PST) Received: by 10.50.183.137 with SMTP id em9mr5418031igc.58.1331166678796; Wed, 07 Mar 2012 16:31:18 -0800 (PST) X-Forwarded-To: linaro-patchwork@canonical.com X-Forwarded-For: patch@linaro.org linaro-patchwork@canonical.com Delivered-To: patches@linaro.org Received: by 10.231.53.18 with SMTP id k18csp25036ibg; Wed, 7 Mar 2012 16:31:18 -0800 (PST) Received: by 10.180.96.230 with SMTP id dv6mr7824937wib.11.1331166677276; Wed, 07 Mar 2012 16:31:17 -0800 (PST) Received: from indium.canonical.com (indium.canonical.com. [91.189.90.7]) by mx.google.com with ESMTPS id o82si20144477weq.132.2012.03.07.16.31.16 (version=TLSv1/SSLv3 cipher=OTHER); Wed, 07 Mar 2012 16:31:17 -0800 (PST) Received-SPF: pass (google.com: best guess record for domain of bounces@canonical.com designates 91.189.90.7 as permitted sender) client-ip=91.189.90.7; Authentication-Results: mx.google.com; spf=pass (google.com: best guess record for domain of bounces@canonical.com designates 91.189.90.7 as permitted sender) smtp.mail=bounces@canonical.com Received: from ackee.canonical.com ([91.189.89.26]) by indium.canonical.com with esmtp (Exim 4.71 #1 (Debian)) id 1S5RGG-0005R0-0s for ; Thu, 08 Mar 2012 00:31:16 +0000 Received: from ackee.canonical.com (localhost [127.0.0.1]) by ackee.canonical.com (Postfix) with ESMTP id F0F54E02CC for ; Thu, 8 Mar 2012 00:31:15 +0000 (UTC) MIME-Version: 1.0 X-Launchpad-Project: lava-server X-Launchpad-Branch: ~linaro-validation/lava-server/trunk X-Launchpad-Message-Rationale: Subscriber X-Launchpad-Branch-Revision-Number: 353 X-Launchpad-Notification-Type: branch-revision To: Linaro Patch Tracker From: noreply@launchpad.net Subject: [Branch ~linaro-validation/lava-server/trunk] Rev 353: add slightly modified version of ajax-backed table code from scheduler Message-Id: <20120308003115.19489.96257.launchpad@ackee.canonical.com> Date: Thu, 08 Mar 2012 00:31:15 -0000 Reply-To: noreply@launchpad.net Sender: bounces@canonical.com Errors-To: bounces@canonical.com Precedence: bulk X-Generated-By: Launchpad (canonical.com); Revision="14907"; Instance="launchpad-lazr.conf" X-Launchpad-Hash: c688e814a082e36abcf74d3c036adcfd369db865 X-Gm-Message-State: ALoCoQnchJ2OVCQ+AfLHWJYFA/M3RVe0izKGhY2ND5WTqQQ9KPtD77ltKwH+LcLdFZTWr5Pf0t/k Merge authors: Michael Hudson-Doyle (mwhudson) Related merge proposals: https://code.launchpad.net/~mwhudson/lava-server/unify-table-code/+merge/96294 proposed by: Michael Hudson-Doyle (mwhudson) review: Approve - Zygmunt Krynicki (zkrynicki) ------------------------------------------------------------ revno: 353 [merge] committer: Michael Hudson-Doyle branch nick: trunk timestamp: Thu 2012-03-08 13:28:53 +1300 message: add slightly modified version of ajax-backed table code from scheduler added: lava/utils/data_tables/tables.py modified: lava/utils/data_tables/backends.py --- lp:lava-server https://code.launchpad.net/~linaro-validation/lava-server/trunk You are subscribed to branch lp:lava-server. To unsubscribe from this branch go to https://code.launchpad.net/~linaro-validation/lava-server/trunk/+edit-subscription === modified file 'lava/utils/data_tables/backends.py' --- lava/utils/data_tables/backends.py 2012-03-01 04:39:00 +0000 +++ lava/utils/data_tables/backends.py 2012-03-07 23:39:48 +0000 @@ -1,6 +1,7 @@ # Copyright (C) 2012 Linaro Limited # # Author: Zygmunt Krynicki +# Author: Michael Hudson-Doyle # # This file is part of LAVA Server. # @@ -18,6 +19,9 @@ from django.core.exceptions import ImproperlyConfigured from django.db.models import Q +from django.template import compile_string + +from django_tables2.rows import BoundRow from lava.utils.data_tables.interface import IBackend @@ -73,6 +77,7 @@ return response + class Column(object): """ Column definition for the QuerySetBackend @@ -152,3 +157,119 @@ for column in self.columns]) for object in queryset] return response + + +simple_nodelist = compile_string('{{ value }}', None) + + +class TableBackend(_BackendBase): + """ + Database backend for data tables. + + Stores and processes/computes the data in the database. + """ + + def __init__(self, table): + self.table = table + + def _render_cell(self, col, data): + """Render data for a column. + + The intent here is to match as precisely as possible the way a cell + would be rendered into HTML by django-tables2. There are two bits of + magic in play here: + + 1) calling the correct render or render_FOO method with the correct + arguments, and + 2) the default Django rendering (escaping if needed, calling unicode() + on model objects etc). + + The first magic is implemented in BoundRow.__getitem__, so we go + through that, and we get the default Django rendering behaviour by + actually rendering what __getitem__ returns in a trivially simple + template (DataTables just sets the innerHTML of the cell to what we + return with no escaping or what have you, so this results in + consistent behaviour). + """ + context = self.table.context + context.update({"value": BoundRow(self.table, data)[col.name]}) + try: + return simple_nodelist.render(context) + finally: + context.pop() + + def _build_q_for_search(self, sSearch): + """Construct a Q object that searches for sSearch. + + The search is split into words, and we return each row that matches + all words. + + A row matches a word if the word appears in (in the sense of + 'icontains') one of the `searchable_columns`. This clearly only + really works for text columns. Extending this to work on + IntegerFields with choices seems possible (you can translate matching + a word into matching the values corresponding to choices whose names + contain the word) but going significantly beyond that would require + some hairy machinery (you could imagine auto generating a model for + each table that stores the rendered rows and searching in that... but + that's mad, surely). + """ + terms = sSearch.split() + andQ = None + for term in terms: + orQ = None + for col in self.table.searchable_columns: + q = Q(**{col+"__icontains" : term}) + orQ = orQ | q if orQ else q + andQ = andQ & orQ if andQ else orQ + return andQ + + def apply_sorting_columns(self, queryset, sorting_columns): + """Sort queryset accoding to sorting_columns. + + `sorting_columns` uses the format used by DataTables, e.g. [[0, + 'asc']] or [[4, 'desc']] or even [[0, 'asc'], [1, 'desc']]. + """ + if not sorting_columns: + return queryset + order_by = [] + for column_index, order in sorting_columns: + col = self.table.columns[column_index] + order_by.append( + "{asc_desc}{column}".format( + asc_desc="-" if order == 'desc' else '', + column=col.accessor.replace('.', '__'))) + return queryset.order_by(*order_by) + + def process(self, query): + """Return the JSON data described by `query`.""" + # Get the basic response structure + response = super(TableBackend, self).process(query) + queryset = self.table.full_queryset + response['iTotalDisplayRecords'] = self.table.full_length + # 1) Apply search/filtering + if query.sSearch: + if query.bRegex: + raise NotImplementedError( + "Searching with regular expresions is not implemented") + else: + if self.table.searchable_columns is None: + raise NotImplementedError("Searching is not implemented") + response['iTotalRecords'] = queryset.count() + queryset = queryset.filter( + self._build_q_for_search(query.sSearch)) + else: + response['iTotalRecords'] = response['iTotalDisplayRecords'] + # TODO: Support per-column search + # 2) Apply sorting + queryset = self.apply_sorting_columns(queryset, query.sorting_columns) + # 3) Apply offset/limit + queryset = queryset[query.iDisplayStart:query.iDisplayStart + query.iDisplayLength] + # Compute the response + # Note, despite the 'aaData' identifier we're + # returing aoData-typed result (array of objects) + response['aaData'] = [ + dict([(column.name, self._render_cell(column, object)) + for column in self.table.columns]) + for object in queryset] + return response === added file 'lava/utils/data_tables/tables.py' --- lava/utils/data_tables/tables.py 1970-01-01 00:00:00 +0000 +++ lava/utils/data_tables/tables.py 2012-03-08 00:11:20 +0000 @@ -0,0 +1,227 @@ +# Copyright (C) 2012 Linaro Limited +# +# Author: Michael Hudson-Doyle +# +# This file is part of LAVA Server. +# +# LAVA Server is free software: you can redistribute it and/or modify +# it under the terms of the GNU Lesser General Public License version 3 +# as published by the Free Software Foundation +# +# LAVA Server is distributed in the hope that it will be useful, +# but WITHOUT ANY WARRANTY; without even the implied warranty of +# MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the +# GNU General Public License for more details. +# +# You should have received a copy of the GNU Lesser General Public License +# along with LAVA Server. If not, see . + +"""Tables designed to be used with the DataTables jQuery plug in. + +There are just three steps to using this: + +1) Define the table:: + + class BookTable(DataTablesTable): + author = Column() + title = Column() + def get_queryset(self): + return Book.objects.all() + +2) Define a view for providing the data from the table in json format:: + + def book_table_json(request): + return BookTable.json(request) + +3) Include the table in the view for the page you are building:: + + def book_list(request): + return render_to_response( + 'my_bookshop_app/book_list.html', + { + 'book_table': BookTable('booklist', reverse(book_table_json)) + }, + RequestContext(request)) + + and in the template:: + + + + {% load django_tables2 %} + + ... + + {% render_table book_table %} + +That's it! + +If the table depends on some parameter, you can pass paramters which end up +getting passed to the get_queryset method. For example:: + + class AuthorBookTable(DataTablesTable): + title = Column() + def get_queryset(self, author): + return author.books + + def author_book_table_json(request, name): + author = get_object_or_404(Author, name=name) + return AuthorBookTable.json(request, (author,)) + + def author_book_list(request, name): + return render_to_response( + author = get_object_or_404(Author, name=name) + 'my_bookshop_app/author_book_list.html', + { + 'author': author, + 'author_book_table': AuthorBookTable( + 'booklist', reverse(author_book_table_json)) + }, + RequestContext(request)) + +In general, usage is designed to be very similar to using the raw +django-tables2 tables. Because the data in the table rendered into html and +in the json view need to be consistent, many of the options that you can pass +to Table's __init__ are not available for DataTablesTable. In practice this +means that you need a DataTablesTable subclass for each different table. +""" + +from abc import ABCMeta, abstractmethod +import simplejson + +from django.template import RequestContext + +from django_tables2.tables import Table +from django_tables2.utils import AttributeDict + +from lava.utils.data_tables.views import DataTableView +from lava.utils.data_tables.backends import TableBackend + + +class MetaTable(ABCMeta, Table.__metaclass__): + pass + + +class DataTablesTable(Table): + """A table designed to be used with the DataTables jQuery plug in. + """ + + __metaclass__ = MetaTable + + def __init__(self, id, source=None, params=(), sortable=None, + empty_text=None, attrs=None, template=None): + """Initialize the table. + + Options that Table supports that affect the data presented are not + supported in this subclass. Extra paramters are: + + :param id: The id of the table in the resulting HTML. You just need + to provide somethign that will be unique in the generated page. + :param source: The URL to get json data from. + :param params: A tuple of arguments to pass to the get_queryset() + method. + """ + if template is None: + template = 'lava_scheduler_app/ajax_table.html' + # The reason we pass data=[] here and patch the queryset in below is + # because of a bootstrapping issue. We want to sort the initial + # queryset, and this is much cleaner if the table has has its .columns + # set up which is only done in Table.__init__... + super(DataTablesTable, self).__init__( + data=[], sortable=sortable, empty_text=empty_text, attrs=attrs, + template=template) + self.full_queryset = self.get_queryset(*params) + self.full_length = self.full_queryset.count() + ordering = self.datatable_opts.get('aaSorting', [[0, 'asc']]) + sorted_queryset = TableBackend(self).apply_sorting_columns( + self.full_queryset, ordering) + display_length = self.datatable_opts.get('iDisplayLength', 10) + del self.data.list + self.data.queryset = sorted_queryset[:display_length] + if source is not None: + self.source = source + # We are careful about modifying the attrs here -- if it comes from + # class Meta:-type options, we don't want to modify the original + # value! + if self.attrs: + attrs = AttributeDict(self.attrs) + else: + attrs = AttributeDict() + attrs.update({ + 'id': id, + # Forcing class to display here is a bit specific really. + 'class': 'display', + }) + self.attrs = attrs + + @classmethod + def json(cls, request, params=()): + """Format table data according to request. + + This method is designed to be called from the view that is passed as + the 'source' argument to a table. The simplest implementation of such + a view would be something like:: + + def table_data(request): + return MyTable.json(request) + + but in general the view might take paramters and pass them as the + `params` argument to this function. + + :param params: A tuple of arguments to pass to the table's + get_queryset() method. + """ + table = cls(None, params=params) + table.context = RequestContext(request) + return DataTableView.as_view( + backend=TableBackend(table) + )(request) + + def datatable_options(self): + """The DataTable options for this table, serialized as JSON.""" + opts = { + 'bJQueryUI': True, + 'bServerSide': True, + 'bProcessing': True, + 'sAjaxSource': self.source, + 'bFilter': bool(self.searchable_columns), + 'iDeferLoading': self.full_length, + } + opts.update(self.datatable_opts) + aoColumnDefs = opts['aoColumnDefs'] = [] + for col in self.columns: + aoColumnDefs.append({ + 'bSortable': bool(col.sortable), + 'mDataProp': col.name, + 'aTargets': [col.name], + }) + return simplejson.dumps(opts) + + # Subclasses must override get_queryset() and may want to provide values + # for source, datatable_opts and searchable_columns. + + @abstractmethod + def get_queryset(self, *args): + """The data the table displays. + + The return data will be sorted, filtered and sliced depending on how + the table is manipulated by the user. + """ + + # The URL to get data from (i.e. the sAjaxSource of the table). Often + # it's more convenient to pass this to the table __init__ to allow the + # code to be laid out in a more logical order. + source = None + + # Extra DataTable options. Values you might want to override here include + # 'iDisplayLength' (how big to make the table's pages by default) and + # 'aaSorting' (the initial sort of the table). See + # http://datatables.net/usage/options for more. + datatable_opts = {} + + # Perform searches by looking in these columns. Searching will not be + # enabled unless you set this. Searching is only supported in textual + # columns for now (supporting an IntegerField with Choices seems possible + # too). + searchable_columns = []