With Early Release ebooks, you get books in their earliest form—the author’s raw and unedited content as they write—so you can take advantage of these technologies long before the official release of these titles.
This will be the 15th chapter of the final book. The GitHub repo is available at https://github.com/hjwp/book-example.
If you have comments about how we might improve the content and/or examples in this book, or if you notice missing material within this chapter, please reach out to the author at obeythetestinggoat@gmail.com.
At the end of the last chapter, we were left with the thought that there was too much duplication in the validation handling bits of our views. Django encourages you to use form classes to do the work of validating user input, and choosing what error messages to display.
We’ll use tests to explore the way Django forms work, and then we’ll refactor our views to use them. As we go along, we’ll see our unit tests and functional tests, in combination, will protect us from regressions.
|
Tip
|
In Django, a complex view is a code smell. Could some of that logic be pushed out to a form? Or to some custom methods on the model class? Or (perhaps best of all) to a non-Django module that represents your business logic? |
Forms have several superpowers in Django:
-
They can process user input and validate it for errors.
-
They can be used in templates to render HTML input elements, and error messages too.
-
And, as we’ll see later, some of them can even save data to the database for you.
You don’t have to use all three form superpowers in every form. You may prefer to roll your own HTML, or do your own saving. But they are an excellent place to keep validation logic.
Let’s do a little experimenting with forms by using a unit test. My plan is to iterate towards a complete solution, and hopefully introduce forms gradually enough that they’ll make sense if you’ve never seen them before.
First we add a new file for our form unit tests, and we start with a test that just looks at the form HTML:
from django.test import TestCase
from lists.forms import ItemForm
class ItemFormTest(TestCase):
def test_form_renders_item_text_input(self):
form = ItemForm()
self.fail(form.as_p())form.as_p() renders the form as HTML. This unit test uses a self.fail
for some exploratory coding. You could just as easily use a manage.py shell
session, although you’d need to keep reloading your code for each change.
Let’s make a minimal form. It inherits from the base Form class, and has
a single field called item_text:
from django import forms
class ItemForm(forms.Form):
item_text = forms.CharField()We now see a failure message which tells us what the autogenerated form HTML will look like:
AssertionError: <p>
<label for="id_item_text">Item text:</label>
<input type="text" name="item_text" required id="id_item_text">
[...]
</p>
It’s already pretty close to what we have in base.html. We’re missing the placeholder attribute and the Bootstrap CSS classes. Let’s make our unit test into a test for that:
class ItemFormTest(TestCase):
def test_form_item_input_has_placeholder_and_css_classes(self):
form = ItemForm()
rendered = form.as_p()
self.assertIn('placeholder="Enter a to-do item"', rendered)
self.assertIn('class="form-control form-control-lg"', rendered)That gives us a fail which justifies some real coding:
self.assertIn('placeholder="Enter a to-do item"', rendered)
~~~~~~~~~~~~~^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
AssertionError: 'placeholder="Enter a to-do item"' not found in [...]
How can we customise the input for a form field? Using a "widget". Here it is with just the placeholder:
class ItemForm(forms.Form):
item_text = forms.CharField(
widget=forms.widgets.TextInput(
attrs={
"placeholder": "Enter a to-do item",
}
),
)That gives:
AssertionError: 'class="form-control form-control-lg"' not found in '<p>\n <label for="id_item_text">Item text:</label>\n <input type="text" name="item_text" placeholder="Enter a to-do item" required id="id_item_text">\n \n \n \n \n </p>'
And then:
widget=forms.widgets.TextInput(
attrs={
"placeholder": "Enter a to-do item",
"class": "form-control form-control-lg",
}
),|
Note
|
Doing this sort of widget customisation would get tedious if we had a much larger, more complex form. Check out django-crispy-forms for some help. |
Does this feel a bit like development-driven tests? That’s OK, now and again.
When you’re exploring a new API, you’re absolutely allowed to mess about with it for a while before you get back to rigorous TDD. You might use the interactive console, or write some exploratory code (but you have to promise the Testing Goat that you’ll throw it away and rewrite it properly later).
Here we’re actually using a unit test as a way of experimenting with the forms API. It can be a surprisingly good way of learning how something works.
What’s next?
We want our form to reuse the validation code that we’ve already defined on our model.
Django provides a special class that can autogenerate a form for a model, called ModelForm.
As you’ll see, it’s configured using a special inner class called Meta:
from django import forms
from lists.models import Item
class ItemForm(forms.models.ModelForm):
class Meta: # (1)
model = Item
fields = ("text",)
# item_text = forms.CharField( #(2)
# widget=forms.widgets.TextInput(
# attrs={
# "placeholder": "Enter a to-do item",
# "class": "form-control form-control-lg",
# }
# ),
# )-
In
Metawe specify which model the form is for, and which fields we want it to use. -
We’ll comment out our manually-created field for now.
ModelForms do all sorts of smart stuff, like assigning sensible HTML form input types to different types of field, and applying default validation. Check out the docs for more info.
We now have some different-looking form HTML:
AssertionError: 'placeholder="Enter a to-do item"' not found in '<p>\n <label for="id_text">Text:</label>\n <textarea name="text" cols="40" rows="10" required id="id_text">\n</textarea>\n \n \n \n \n </p>'
It’s lost our placeholder and CSS class.
And you can also see that it’s using
name="text" instead of name="item_text".
We can probably live with that.
But it’s using a textarea instead of a normal input,
and that’s not the UI we want for our app.
Thankfully, you can override widgets for ModelForm fields,
similarly to the way we did it with the normal form:
class ItemForm(forms.models.ModelForm):
class Meta:
model = Item
fields = ("text",)
widgets = { # (1)
"text": forms.widgets.TextInput(
attrs={
"placeholder": "Enter a to-do item",
"class": "form-control form-control-lg",
}
),
}-
We restore some of our commented-out code here, but modified slightly, from being an attribute declaration, to a key in a dict.
That gets the test passing.
Now let’s see if the ModelForm has picked up the same validation rules
which we defined on the model.
We’ll also learn how to pass data into the form, as if it came from the user:
def test_form_item_input_has_placeholder_and_css_classes(self):
[...]
def test_form_validation_for_blank_items(self):
form = ItemForm(data={"text": ""})
form.save()That gives us:
ValueError: The Item could not be created because the data didn't validate.
Good: the form won’t allow you to save if you give it an empty item text.
Now let’s see if we can get it to use the specific error message that we
want. The API for checking form validation 'before' we try to save any
data is a function called is_valid:
def test_form_item_input_has_placeholder_and_css_classes(self):
[...]
def test_form_validation_for_blank_items(self):
[...]
def test_form_validation_for_blank_items(self):
form = ItemForm(data={"text": ""})
self.assertFalse(form.is_valid())
self.assertEqual(form.errors["text"], ["You can't have an empty list item"])Calling form.is_valid() returns True or False,
but it also has the side effect of validating the input data,
and populating the errors attribute.
It’s a dictionary mapping the names of fields to lists of errors for those fields
(it’s possible for a field to have more than one error).
That gives us:
AssertionError: ['This field is required.'] != ["You can't have an empty list item"]
Django already has a default error message
that we could present to the user—you might use it
if you were in a hurry to build your web app,
but we care enough to make our message special.
Customising it means changing error_messages, another Meta variable:
class Meta:
model = Item
fields = ("text",)
widgets = {
"text": forms.widgets.TextInput(
attrs={
"placeholder": "Enter a to-do item",
"class": "form-control form-control-lg",
}
),
}
error_messages = {"text": {"required": "You can't have an empty list item"}}OK
You know what would be even better than messing about with all these error strings? Having a constant:
EMPTY_ITEM_ERROR = "You can't have an empty list item"
[...]
error_messages = {"text": {"required": EMPTY_ITEM_ERROR}}Rerun the tests to see that they pass…OK. Now we can change the tests too.
from lists.forms import EMPTY_ITEM_ERROR, ItemForm
[...]
def test_form_validation_for_blank_items(self):
form = ItemForm(data={"text": ""})
self.assertFalse(form.is_valid())
self.assertEqual(form.errors["text"], [EMPTY_ITEM_ERROR])|
Tip
|
This is a good example of re-using constants in tests. It makes it easier to change the error message later. |
And the tests still pass:
OK
Great. Totes committable:
$ git status # should show forms.py and test_forms.py $ git add src/lists $ git commit -m "new form for list items"
At this point we may be tempted to carry on, perhaps extend the form to capture uniqueness validation as well as empty-item validation.
But there’s a sort of corollary to the "deploy as early as possible" lean methodology, which is "merge code as early as possible". In other words: while building this bit of forms code, it would be easy to go on for ages, adding more and more functionality to the form—I should know, because that’s exactly what I did during the drafting of this chapter, and I ended up doing all sorts of work making an all-singing, all-dancing form class before I realised it wouldn’t actually work for our most basic use case.
So, instead, try to use your new bit of code as soon as possible. This makes sure you never have unused bits of code lying around, and that you start checking your code against "the real world" as soon as possible.
We have a form class that can render some HTML and do validation of at least one kind of error—let’s start using it! We should be able to use it in our base.html template, and so in all of our views.
So let’s start using our form in our home page view:
[...]
from lists.forms import ItemForm
from lists.models import Item, List
def home_page(request):
return render(request, "home.html", {"form": ItemForm()})OK, now let’s try using it in the template—we
replace the old <input ..> with {{ form.text }}:
<form method="POST" action="{% block form_action %}{% endblock %}" >
{{ form.text }} (1)
{% csrf_token %}
{% if error %}
<div class="invalid-feedback">{{ error }}</div>
{% endif %}
</form>-
{{ form.text }}renders just the HTML input for thetextfield of the form.
That causes our two unit tests that check on the form input to fail:
[...]
======================================================================
FAIL: test_renders_input_form
(lists.tests.test_views.HomePageTest.test_renders_input_form)
---------------------------------------------------------------------
Traceback (most recent call last):
File "...goat-book/src/lists/tests/test_views.py", line 19, in
test_renders_input_form
self.assertIn("item_text", [input.get("name") for input in inputs])
~~~~~~~~~~~~~^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
AssertionError: 'item_text' not found in ['text', 'csrfmiddlewaretoken'] (1)
======================================================================
FAIL: test_renders_input_form
(lists.tests.test_views.ListViewTest.test_renders_input_form)
---------------------------------------------------------------------
Traceback (most recent call last):
File "...goat-book/src/lists/tests/test_views.py", line 60, in
test_renders_input_form
self.assertIn("item_text", [input.get("name") for input in inputs])
~~~~~~~~~~~~~^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
AssertionError: 'item_text' not found in ['csrfmiddlewaretoken'] (2)
Ran 18 tests in 0.022s
FAILED (failures=2)
-
The test for the home page is failing because the
nameattribute of the input box is nowtext, notitem_text. -
The test for the list view is failing because because we’re not instantiating a form in that view, so there’s on
formvariable in the template, the input box is not even being rendered at all,
Let’s fix things one at a time.
First, let’s back out our change and restore the hand-coded html input
in cases where {{ form }} is not defined:
<form method="POST" action="{% block form_action %}{% endblock %}" >
{% if form %}
{{ form.text }}
{% else %}
<input
class="form-control form-control-lg {% if error %}is-invalid{% endif %}"
name="item_text"
id="id_new_item"
placeholder="Enter a to-do item"
/>
{% endif %}
{% csrf_token %}
{% if error %}
<div class="invalid-feedback">{{ error }}</div>
{% endif %}
</form>That takes us down to 1 failure:
AssertionError: 'item_text' not found in ['text', 'csrfmiddlewaretoken']
Let’s make a note to come back and tidy this up, then talk about what’s happened and how to deal with it.
-
Remove duplication of validation logic in views
-
Remove if branch and hardcoded input tag from base.html
This highlights one of the tradeoffs of using ModelForm:
by auto-generating the form from the model,
we tie the name= attribute of our form’s HTML <input>
to the name of the model field in the database.
In a simple CRUD app like ours, that’s probably a good deal.
But it does mean we need to go back and change our assumptions about
what the name= attribute of the input box is going to be.
While we’re at it, it’s worth doing an FT run too:
$ python src/manage.py test functional_tests [...] selenium.common.exceptions.NoSuchElementException: Message: Unable to locate element: [id="id_new_item"]; [...] [...] FAILED (errors=4)
Looks like something else has changed.
If you pause the FTs or inspect the HTML manually in a browser,
you’ll see that the ModelForm also changes the id attribute
to being id_text.[1].
If we want to change our assumption about these two attributes, we’ll need to embark on a couple of big find and replaces basically:
-
Remove duplication of validation logic in views
-
Remove if branch and hardcoded input tag from base.html
-
change input name attribute from item_text to just text
-
change input id from id_new_item to id_text
But before we do that, let’s back out the rest of our changes and get back to a working state.
The simplest way to back out changes is with git.
But in this case, leaving a couple of placeholders does no harm,
and they’ll be helpful to come back to later.
So we can leave the {{ form.text }} in the HTML,
but we’ll make sure that branch is never actually exercised,
by backing out the change in the view.
Again, to leave ourselves a little placeholder,
we’ll comment out our code rather than deleting it.
def home_page(request):
# return render(request, "home.html", {"form": ItemForm()})
return render(request, "home.html")|
Warning
|
Be very cautious about leaving commented-out code and unused if branches lying around. Do so only if you’re sure you’re coming back to them very soon, otherwise your codebase will soon get messy! |
Now we can do a full unit tests and FT run to confirm we’re back to a working state:
$ python src/manage.py test lists Found 18 test(s). [...] OK $ python src/manage.py test functional_tests Found 4 test(s). [...] OK
And let’s do a commit to be able to separate out the rename from anything else:
$ git diff # changes in base.html + views.py $ git commit -am "Placeholders for using form in view+template, not in use yet"
And pop an item on the todo list
-
Remove duplication of validation logic in views
-
Remove if branch and hardcoded input tag from base.html
-
change input name attribute from item_text to just text
-
change input id from id_new_item to id_text
-
uncomment use of form in home_page() view_item to id_text
-
use form in other views
So, let’s have a look for item_text in the codebase.
$ grep -Ir item_text src
src/lists/migrations/0003_list.py: ("lists", "0002_item_text"),
src/lists/tests/test_views.py: self.assertIn("item_text",
[input.get("name") for input in inputs])
[...]
src/lists/templates/base.html: name="item_text"
src/lists/views.py: item = Item(text=request.POST["item_text"], list=nulist)
src/lists/views.py: item = Item(text=request.POST["item_text"],
list=our_list)
We can ignore the migration which is just using item_text as metadata.
So the changes we need to make are in three places:
-
views.py
-
test_views.py
-
base.html
Let’s go ahead and make those. I’m sure you can manage your own find and replace! They should look something like this:
@@ -16,12 +16,12 @@ class HomePageTest(TestCase):
[form] = parsed.cssselect("form[method=POST]")
self.assertEqual(form.get("action"), "/lists/new")
inputs = form.cssselect("input")
- self.assertIn("item_text", [input.get("name") for input in inputs])
+ self.assertIn("text", [input.get("name") for input in inputs])
class NewListTest(TestCase):
def test_can_save_a_POST_request(self):
- self.client.post("/lists/new", data={"item_text": "A new list item"})
+ self.client.post("/lists/new", data={"text": "A new list item"})
self.assertEqual(Item.objects.count(), 1)
new_item = Item.objects.get()
self.assertEqual(new_item.text, "A new list item")
[...]Or, in views.py:
@@ -12,7 +12,7 @@ def home_page(request):
def new_list(request):
nulist = List.objects.create()
- item = Item(text=request.POST["item_text"], list=nulist)
+ item = Item(text=request.POST["text"], list=nulist)
try:
item.full_clean()
item.save()
@@ -29,7 +29,7 @@ def view_list(request, list_id):
if request.method == "POST":
try:
- item = Item(text=request.POST["item_text"], list=our_list)
+ item = Item(text=request.POST["text"], list=our_list)
item.full_clean()
item.save()
return redirect(our_list)Finally, in base.html:
@@ -21,7 +21,7 @@
{% else %}
<input
class="form-control form-control-lg {% if error %}is-invalid{% endif %}"
- name="item_text"
+ name="text"
id="id_new_item"
placeholder="Enter a to-do item"
/>Once you’re done, rerun the unit tests to confirm that the application is self-consistent:
$ python src/manage.py test lists [...] Ran 18 tests in 0.126s OK
And rerun the FTs too:
$ python src/manage.py test functional_tests [...] Ran 4 tests in 12.154s OK
Good! One down:
-
Remove duplication of validation logic in views
-
Remove if branch and hardcoded input tag from base.html
-
change input name attribute from item_text to just text
-
change input id from id_new_item to id_text
-
uncomment use of form in home_page() view_item to id_text
-
use form in other views
Now for the id= attribute.
A quick grep shows us that id_new_item appears in the template,
and in all 3 FT files:
$ grep -r id_new_item
src/lists/templates/base.html: id="id_new_item"
src/functional_tests/test_list_item_validation.py:
self.browser.find_element(By.ID, "id_new_item").send_keys(Keys.ENTER)
src/functional_tests/test_list_item_validation.py:
self.browser.find_element(By.ID, "id_new_item").send_keys("Purchase milk")
[...]
That’s a good call for a refactor within the FTs too. Let’s make a new helper method in base.py:
class FunctionalTest(StaticLiveServerTestCase):
[...]
def get_item_input_box(self):
return self.browser.find_element(By.ID, "id_new_item") # (1)-
We’ll keep the old ID for now. Working state to working state!
And then we use it throughout—I had to make four changes in test_simple_list_creation.py, two in test_layout_and_styling.py, and six in test_list_item_validation.py, for example:
# She is invited to enter a to-do item straight away
inputbox = self.get_item_input_box()Or:
# an empty list item. She hits Enter on the empty input box
self.browser.get(self.live_server_url)
self.get_item_input_box().send_keys(Keys.ENTER)I won’t show you every single one; I’m sure you can manage this for yourself!
You can redo the grep to check that you’ve caught them all:
$ grep -r id_new_item src/lists/templates/base.html: id="id_new_item" src/functional_tests/base.py: return self.browser.find_element(By.ID, "id_new_item")
And we can do an FT run too, to make sure we haven’t broken anything:
$ python src/manage.py test functional_tests [...] Ran 4 tests in 12.154s OK
Good! FT refactor complete, now now hopefully we can make
the application-level refactor of the id attribute in just two places,
and we’ve been in a working state the whole way through.
In the FT helper method:
@@ -43,4 +43,4 @@ class FunctionalTest(StaticLiveServerTestCase):
time.sleep(0.5)
def get_item_input_box(self):
- return self.browser.find_element(By.ID, "id_new_item")
+ return self.browser.find_element(By.ID, "id_text")And in the template:
@@ -22,7 +22,7 @@
<input
class="form-control form-control-lg {% if error %}is-invalid{% endif %}"
name="text"
- id="id_new_item"
+ id="id_text"
placeholder="Enter a to-do item"
/>
{% endif %}And an FT run to confirm:
$ python src/manage.py test functional_tests [...] Ran 4 tests in 12.154s OK
Hooray!
-
Remove duplication of validation logic in views
-
Remove if branch and hardcoded input tag from base.html
-
change input name attribute from item_text to just text
-
change input id from id_new_item to id_text
-
uncomment use of form in home_page() view_item to id_text
-
use form in other views
Now we’ve done the groundwork,
hopefully we can drop in our form in the home_page() once again:
def home_page(request):
return render(request, "home.html", {"form": ItemForm()})Looking good!
$ python src/manage.py test lists Found 18 test(s). [...] OK
-
Remove duplication of validation logic in views
-
Remove if branch and hardcoded input tag from base.html
-
change input name attribute from item_text to just text
-
change input id from id_new_item to id_text
-
uncomment use of form in home_page() view_item to id_text
-
use form in other views
Let’s see what happens if we remove that if from the template:
@@ -16,16 +16,7 @@
<h1 class="display-1 mb-4">{% block header_text %}{% endblock %}</h1>
<form method="POST" action="{% block form_action %}{% endblock %}" >
- {% if form %}
- {{ form.text }}
- {% else %}
- <input
- class="form-control form-control-lg {% if error %}is-invalid{% endif %}"
- name="text"
- id="id_text"
- placeholder="Enter a to-do item"
- />
- {% endif %}
+ {{ form.text }}
{% csrf_token %}
{% if error %}
<div class="invalid-feedback">{{ error }}</div>Aha, the unit tests are there to tell us
that we need to the form in view_list() too:
AssertionError: 'text' not found in ['csrfmiddlewaretoken']
Here’s the minimal use of the form—we won’t use it for validation yet, just for getting the form into the template:
def view_list(request, list_id):
our_list = List.objects.get(id=list_id)
error = None
form = ItemForm()
if request.method == "POST":
try:
item = Item(text=request.POST["text"], list=our_list)
item.full_clean()
item.save()
return redirect(our_list)
except ValidationError:
error = "You can't have an empty list item"
return render(
request, "list.html", {"list": our_list, "form": form, "error": error}
)And the tests are happy with that too:
$ python src/manage.py test lists Found 18 test(s). [...] OK
We’re done with the template, what’s next?
-
Remove duplication of validation logic in views
-
Remove if branch and hardcoded input tag from base.html
-
change input name attribute from item_text to just text
-
change input id from id_new_item to id_text
-
uncomment use of form in home_page() view_item to id_text
-
use form in other views
Right, let’s move on to the next view that doesn’t use our form yet, new_list(),
And actually, that’ll help us with the first item,
which was the whole point of this adventure really,
see if the forms can help us do validation more nicely.
Let’s see how that works now.
Here’s how we can use the form in the new_list() view,
avoiding all the manual manipulation of request.POST and the error message:
def new_list(request):
form = ItemForm(data=request.POST) #(1)
if form.is_valid(): #(2)
nulist = List.objects.create()
Item.objects.create(text=request.POST["text"], list=nulist)
return redirect(nulist)
else:
return render(request, "home.html", {"form": form}) #(3)-
We pass the
request.POSTdata into the form’s constructor. -
We use
form.is_valid()to determine whether this is a good or a bad submission. -
In the invalid case, we pass the form down to the template, instead of our hardcoded error string.
That view is now looking much nicer!
But, we have a regression in the unit tests:
======================================================================
FAIL: test_validation_errors_are_sent_back_to_home_page_template (lists.tests.t
est_views.NewListTest.test_validation_errors_are_sent_back_to_home_page_templat
e)
---------------------------------------------------------------------
[...]
self.assertContains(response, expected_error)
~~~~~~~~~~~~~~~~~~~^^^^^^^^^^^^^^^^^^^^^^^^^^
AssertionError: False is not true : Couldn't find 'You can't have an empty
list item' in the following response
b'<!doctype html>\n<html lang="en">\n\n <head>\n <title>To-Do
[...]
We’re failing because we’re not yet using the form to display errors in the template. Here’s how to do that:
<form method="POST" action="{% block form_action %}{% endblock %}" >
{{ form.text }}
{% csrf_token %}
{% if form.errors %} (1)
<div class="invalid-feedback">{{ form.errors.text }}</div> (2)
{% endif %}
</form>-
We change the
ifto look atform.errors: it contains a list of all the errors for the form. -
form.errors.textis magical Django template syntax forform.errors["text"], i.e., the list of errors for the text field in particular.
What does that do to our unit tests?
====================================================================== FAIL: test_validation_errors_end_up_on_lists_page (lists.tests.test_views.ListV iewTest.test_validation_errors_end_up_on_lists_page) --------------------------------------------------------------------- [...] AssertionError: False is not true : Couldn't find 'You can't have an empty list item' in the following response
An unexpected failure—it’s actually in the tests for our final view, view_list().
Once again, because we’ve changed the base template which is used by _all_views, we’ve made a change that impacts more places than we intended.
Let’s follow our standard pattern, get back to a working state, and see if we can dig into this a bit.
Let’s restore the old [% if %} in the template,
so we display errors in both old+new cases:
<form method="POST" action="{% block form_action %}{% endblock %}" >
{{ form.text }}
{% csrf_token %}
{% if error %}
<div class="invalid-feedback">{{ error }}</div>
{% endif %}
{% if form.errors %}
<div class="invalid-feedback">{{ form.errors.text }}</div>
{% endif %}
</form>And add an item to our stack:
-
Remove duplication of validation logic in views
-
Remove if branch and hardcoded input tag from base.html
-
change input name attribute from item_text to just text
-
change input id from id_new_item to id_text
-
uncomment use of form in home_page() view_item to id_text
-
use form in other views
-
remove if error branch from template
Let’s take a look at our tests for both views, particularly the ones that check for invalid inputs:
class NewListTest(TestCase):
[...]
def test_validation_errors_are_sent_back_to_home_page_template(self):
response = self.client.post("/lists/new", data={"text": ""})
self.assertEqual(response.status_code, 200)
self.assertTemplateUsed(response, "home.html")
expected_error = html.escape("You can't have an empty list item")
self.assertContains(response, expected_error)
def test_invalid_list_items_arent_saved(self):
self.client.post("/lists/new", data={"text": ""})
self.assertEqual(List.objects.count(), 0)
self.assertEqual(Item.objects.count(), 0)
class ListViewTest(TestCase):
[...]
def test_validation_errors_end_up_on_lists_page(self):
list_ = List.objects.create()
response = self.client.post(
f"/lists/{list_.id}/",
data={"text": ""},
)
self.assertEqual(response.status_code, 200)
self.assertTemplateUsed(response, "list.html")
expected_error = html.escape("You can't have an empty list item")
self.assertContains(response, expected_error)I see a few problems here:
-
We’re check explicitly that errors mean nothing is saved to the DB in
NewListTestbut not inListViewTest. -
We’re mixing up the test for the status code, the template, and finding the error in the result.
Let’s be extra meticulous here, and separate out these concerns. Ideally each test should have one assert. If we used copy-paste, that would start to involve a lot of duplication, so using a couple of helper methods is a good idea here.
Here’s some better tests in NewListTest:
from lists.forms import EMPTY_ITEM_ERROR
[...]
class NewListTest(TestCase):
def test_can_save_a_POST_request(self):
[...]
def test_redirects_after_POST(self):
[...]
def post_invalid_input(self):
return self.client.post("/lists/new", data={"text": ""})
def test_for_invalid_input_nothing_saved_to_db(self):
self.post_invalid_input()
self.assertEqual(Item.objects.count(), 0)
def test_for_invalid_input_renders_list_template(self):
response = self.post_invalid_input()
self.assertEqual(response.status_code, 200)
self.assertTemplateUsed(response, "home.html")
def test_for_invalid_input_shows_error_on_page(self):
response = self.post_invalid_input()
self.assertContains(response, html.escape(EMPTY_ITEM_ERROR))By making a little helper function, post_invalid_input(),
we can make three separate tests without duplicating lots of lines of code.
We’ve seen this several times now. It often feels more natural to write view tests as a single, monolithic block of assertions—the view should do this and this and this, then return that with this. But breaking things out into multiple tests is often worthwhile; as we saw in previous chapters, it helps you isolate the exact problem you have when you later accidentally introduce a bug. Helper methods are one of the tools that lower the psychological barrier, by reducing boilerplate and keeping the tests readable.
Let’s do something similar in ListViewTest:
class ListViewTest(TestCase):
def test_uses_list_template(self):
[...]
def test_renders_input_form(self):
[...]
def test_displays_only_items_for_that_list(self):
[...]
def test_can_save_a_POST_request_to_an_existing_list(self):
[...]
def test_POST_redirects_to_list_view(self):
[...]
def post_invalid_input(self):
mylist = List.objects.create()
return self.client.post(f"/lists/{mylist.id}/", data={"text": ""})
def test_for_invalid_input_nothing_saved_to_db(self):
self.post_invalid_input()
self.assertEqual(Item.objects.count(), 0)
def test_for_invalid_input_renders_list_template(self):
response = self.post_invalid_input()
self.assertEqual(response.status_code, 200)
self.assertTemplateUsed(response, "list.html")
def test_for_invalid_input_shows_error_on_page(self):
response = self.post_invalid_input()
self.assertContains(response, html.escape(EMPTY_ITEM_ERROR))And let’s re-run all our tests:
$ python src/manage.py test lists Found 21 test(s). [...] OK
Great! We now feel confident that we have a lot of very specific unit tests, that can point us to exactly what goes wrong if we ever make a mistake.
So let’s have another go at using our form for all views,
by fully committing to the {{ form.errors }} in the template:
@@ -18,9 +18,6 @@
<form method="POST" action="{% block form_action %}{% endblock %}" >
{{ form.text }}
{% csrf_token %}
- {% if error %}
- <div class="invalid-feedback">{{ error }}</div>
- {% endif %}
{% if form.errors %}
<div class="invalid-feedback">{{ form.errors.text }}</div>
{% endif %}And we’ll see that exactly one test is failing:
FAIL: test_for_invalid_input_shows_error_on_page (lists.tests.test_views.ListVi ewTest.test_for_invalid_input_shows_error_on_page) [...] AssertionError: False is not true : Couldn't find 'You can't have an empty list item' in the following response
Let’s try and work step by step towards fully using our form in this final view.
At the moment one test is failing because,
in the existing lists view_list() view,
we aren’t passing populating form.errors in the invalid case.
So let’s address just that:
def view_list(request, list_id):
our_list = List.objects.get(id=list_id)
error = None
form = ItemForm() # (2)
if request.method == "POST":
form = ItemForm(data=request.POST) # (1)
try:
item = Item(text=request.POST["text"], list=our_list)
item.full_clean()
item.save()
return redirect(our_list)
except ValidationError:
error = "You can't have an empty list item"
return render(
request, "list.html", {"list": our_list, "form": form, "error": error} # (3)
)-
Let’s add this line, in the
method=POSTbranch, and instantiate a form using the POST data. -
We already had this empty form for the GET case, but our new one will override it.
-
And it should now drop through to the template here.
That gets us back to a working state!
Found 21 test(s). [...] OK
-
Remove duplication of validation logic in views
-
Remove if branch and hardcoded input tag from base.html
-
change input name attribute from item_text to just text
-
change input id from id_new_item to id_text
-
uncomment use of form in home_page() view_item to id_text
-
use form in other views
-
remove if error branch from template
Now let’s start using the form more fully, and remove some of the manual error handling.
We remove the try/except and replace it with an
if form.is_valid() check, like the one in new_list():
@@ -26,13 +26,11 @@ def view_list(request, list_id):
if request.method == "POST":
form = ItemForm(data=request.POST)
- try:
+ if form.is_valid():
item = Item(text=request.POST["text"], list=our_list)
item.full_clean()
item.save()
return redirect(our_list)
- except ValidationError:
- error = "You can't have an empty list item"
return render(
request, "list.html", {"list": our_list, "form": form, "error": error}And the tests still pass:
OK
Next, we no longer need the .full_clean(),
so we can go back to using .objects.create():
@@ -27,9 +27,7 @@ def view_list(request, list_id):
if request.method == "POST":
form = ItemForm(data=request.POST)
if form.is_valid():
- item = Item(text=request.POST["text"], list=our_list)
- item.full_clean()
- item.save()
+ Item.objects.create(text=request.POST["text"], list=our_list)
return redirect(our_list)Tests still pass:
OK
Finally, the error variable is always None,
and is no longer needed in the template anyhow:
@@ -21,7 +21,6 @@ def new_list(request):
def view_list(request, list_id):
our_list = List.objects.get(id=list_id)
- error = None
form = ItemForm()
if request.method == "POST":
@@ -30,6 +29,4 @@ def view_list(request, list_id):
Item.objects.create(text=request.POST["text"], list=our_list)
return redirect(our_list)
- return render(
- request, "list.html", {"list": our_list, "form": form, "error": error}
- )
+ return render(request, "list.html", {"list": our_list, "form": form})And the tests are happy with that!
OK
I think our view is in a pretty good shape now. Here it is in non-diff mode, as a recap:
def view_list(request, list_id):
our_list = List.objects.get(id=list_id)
form = ItemForm()
if request.method == "POST":
form = ItemForm(data=request.POST)
if form.is_valid():
Item.objects.create(text=request.POST["text"], list=our_list)
return redirect(our_list)
return render(request, "list.html", {"list": our_list, "form": form})I think we can give ourselves the satisfaction of doing some crossing-things-out:
-
[strikethrough line-through]Remove duplication of validation logic in views#
-
Remove if branch and hardcoded input tag from base.html
-
change input name attribute from item_text to just text
-
change input id from id_new_item to id_text
-
uncomment use of form in home_page() view_item to id_text
-
[strikethrough line-through]use form in other views#
-
remove if error branch from template
Phew!
Hey, it’s been a while, what do our FTs think?
[...] ====================================================================== ERROR: test_cannot_add_empty_list_items (functional_tests.test_list_item_valida tion.ItemValidationTest.test_cannot_add_empty_list_items) --------------------------------------------------------------------- [...] selenium.common.exceptions.NoSuchElementException: Message: Unable to locate element: .invalid-feedback; [...] [...] Ran 4 tests in 14.897s FAILED (errors=1)
Oh. All the regression tests are OK, but our validation test seems to be failing, and failing early too! It’s on the first attempt to submit an empty item. What happened?
How shall we find out what’s going on here?
One option is to add the usual time.sleep just before the error in the FTs,
and take a look at what’s happening while they run.
Alternatively, spin up the site manually with manage.py runserver if you prefer.
Either way, you should see something like HTML5 validation says no.
It seems like the browser is preventing the user from even submitting the input when it’s empty.
It’s because Django has added the required attribute to the HTML input
(take another look at our as_p() printouts from earlier if you don’t believe me,
or have a look at the source in Dev tools).
This is a feature of HTML5; browsers nowadays will do some validation at the client side if they can, preventing users from even submitting invalid input. That’s actually good news!
But, we were working based on incorrect assumptions about what the user experience was going to be. Let’s change our FT to reflect this new expectation.
class ItemValidationTest(FunctionalTest):
def test_cannot_add_empty_list_items(self):
# Edith goes to the home page and accidentally tries to submit
# an empty list item. She hits Enter on the empty input box
self.browser.get(self.live_server_url)
self.get_item_input_box().send_keys(Keys.ENTER)
# The browser intercepts the request, and does not load the list page
self.wait_for(
lambda: self.browser.find_element(By.CSS_SELECTOR, "#id_text:invalid") #(1)
)
# She starts typing some text for the new item and the error disappears
self.get_item_input_box().send_keys("Purchase milk")
self.wait_for(
lambda: self.browser.find_element(By.CSS_SELECTOR, "#id_text:valid") #(2)
)
# And she can submit it successfully
self.get_item_input_box().send_keys(Keys.ENTER)
self.wait_for_row_in_list_table("1: Purchase milk")
# Perversely, she now decides to submit a second blank list item
self.get_item_input_box().send_keys(Keys.ENTER)
# Again, the browser will not comply
self.wait_for_row_in_list_table("1: Purchase milk")
self.wait_for(
lambda: self.browser.find_element(By.CSS_SELECTOR, "#id_text:invalid")
)
# And she can make it happy by filling some text in
self.get_item_input_box().send_keys("Make tea")
self.wait_for(
lambda: self.browser.find_element(
By.CSS_SELECTOR,
"#id_text:valid",
)
)
self.get_item_input_box().send_keys(Keys.ENTER)
self.wait_for_row_in_list_table("2: Make tea")-
Instead of checking for our custom error message, we check using the CSS pseudoselector
:invalid, which the browser applies to any HTML5 input that has invalid input. -
And its converse in the case of valid inputs.
See how useful and flexible our self.wait_for() function is turning out to be?
Our FT does look quite different from how it started though, doesn’t it? I’m sure that’s raising a lot of questions in your mind right now. Put a pin in them for a moment; I promise we’ll talk. Let’s first see if we’re back to passing tests:
$ python src/manage.py test functional_tests [...] Ran 4 tests in 12.154s OK
First let’s give ourselves a massive pat on the back: we’ve just made a major change to our small app—that input field, with its name and ID, is absolutely critical to making everything work. We’ve touched seven or eight different files, doing a refactor that’s quite involved…this is the kind of thing that, without tests, would seriously worry me. In fact, I might well have decided that it wasn’t worth messing with code that works. But, because we have a full test suite, we can delve around, tidying things up, safe in the knowledge that the tests are there to spot any mistakes we make. It just makes it that much likelier that you’re going to keep refactoring, keep tidying up, keep gardening, keep tending your code, keep everything neat and tidy and clean and smooth and precise and concise and functional and good.
And it’s definitely time for a commit:
$ git diff $ git commit -am "use form in all views, back to working state"
But what about our custom error message? What about all that effort rendering the form in our HTML template? We’re not even passing those errors from Django to the user if the browser is intercepting the requests before the user even makes them? And our FT isn’t even testing that stuff any more!
Well, you’re quite right. But there are two or three reasons all our time hasn’t been wasted. Firstly, client-side validation isn’t enough to guarantee you’re protected from bad inputs, so you always need the server side as well if you really care about data integrity; using a form is a nice way of encapsulating that logic.
Also, not all browsers fully implement HTML5,[2] so some users might still see our custom error message. And if or when we come to letting users access our data via an API (see Online Appendix: Building a REST API), then our validation messages will come back into use.
On top of that, we’ll be able to reuse all our validation and forms code when we do some more advanced validation that can’t be done by HTML5 magic.
But you know, even if all that wasn’t true, you still can’t beat yourself up for occasionally going down a blind alley while you’re coding. None of us can see the future, and we should concentrate on finding the right solution rather than the time "wasted" on the wrong solution.
There are a couple more things we can do to make our views even simpler. I’ve mentioned that forms are supposed to be able to save data to the database for us. Our case won’t quite work out of the box, because the item needs to know what list to save to, but it’s not hard to fix that.
We start, as always, with a test.
Just to illustrate what the problem is,
let’s see what happens if we just try to call form.save():
def test_form_save_handles_saving_to_a_list(self):
form = ItemForm(data={"text": "do me"})
new_item = form.save()Django isn’t happy, because an item needs to belong to a list:
django.db.utils.IntegrityError: NOT NULL constraint failed: lists_item.list_id
Our solution is to tell the form’s save method what list it should save to.
from lists.models import Item, List
[...]
def test_form_save_handles_saving_to_a_list(self):
mylist = List.objects.create()
form = ItemForm(data={"text": "do me"})
new_item = form.save(for_list=mylist) # (1)
self.assertEqual(new_item, Item.objects.get()) #(2)
self.assertEqual(new_item.text, "do me")
self.assertEqual(new_item.list, mylist)-
We’ll imagine that the
.save()method takes afor_list=argument. -
We then make sure that the item is correctly saved to the database, with the right attributes.
The tests fail as expected, because as usual, it’s still only wishful thinking:
new_item = form.save(for_list=mylist) TypeError: BaseModelForm.save() got an unexpected keyword argument 'for_list'
Here’s how we can implement a custom save method:
class ItemForm(forms.models.ModelForm):
class Meta:
[...]
def save(self, for_list):
self.instance.list = for_list
return super().save()The .instance attribute on a form represents the database object
that is being modified or created.
And I only learned that as I was writing this chapter!
There are other ways of getting this to work,
including manually creating the object yourself,
or using the commit=False argument to save,
but this way seemed neatest.
We’ll explore a different way of making a form "know" what list it’s for
in the next chapter.
Ran 22 tests in 0.086s OK
Finally, we can refactor our views. new_list() first:
def new_list(request):
form = ItemForm(data=request.POST)
if form.is_valid():
nulist = List.objects.create()
form.save(for_list=nulist)
return redirect(nulist)
else:
return render(request, "home.html", {"form": form})Rerun the test to check that everything still passes:
Ran 22 tests in 0.086s OK
Then, refactor view_list():
def view_list(request, list_id):
our_list = List.objects.get(id=list_id)
form = ItemForm()
if request.method == "POST":
form = ItemForm(data=request.POST)
if form.is_valid():
form.save(for_list=our_list)
return redirect(our_list)
return render(request, "list.html", {"list": our_list, "form": form})We still have full passes:
Ran 22 tests in 0.111s OK
and:
Ran 4 tests in 14.367s OK
Great! Let’s commit our changes:
$ git commit -am "implement custom save method for the form"
Our two views are now looking very much like "normal" Django views:
they take information from a user’s request,
combine it with some custom logic or information from the URL (list_id),
pass it to a form for validation and possible saving,
and then redirect or render a template.
Forms and validation are really important in Django, and in web programming in general, so let’s try to make a slightly more complicated one in the next chapter, to learn how to prevent duplicate items.
- Thin views
-
If you find yourself looking at complex views, and having to write a lot of tests for them, it’s time to start thinking about whether that logic could be moved elsewhere: possibly to a form, like we’ve done here.
Another possible place would be a custom method on the model class. And—once the complexity of the app demands it—out of Django-specific files and into your own classes and functions, that capture your core business logic. - Each test should test one thing
-
The heuristic is to be suspicious if there’s more than one assertion in a unit test. Sometimes two assertions are closely related, so they belong together. But often your first draft of a test ends up testing multiple behaviours, and it’s worth rewriting it as several tests, so that each one can pinpoint specific problems more precisely, and one failure doesn’t mask another. Helper functions can keep them your tests getting too bloated.
- Be aware of tradeoffs when using frameworks
-
When we switched to using a ModelForm, we saw that it forced us to change the
name=attribute in our frontend HTML. Django gave us a lot, it auto-generated the form based on the model, and we have a nice API for doing both validation and saving objects. But we lost something too. We’ll revisit this tradeoff in the next chapter.