On Properties, part 2

⏰ Takes about 9 minutes to read

A couple weeks back I wrote about this sort of strange topic called Property Based Testing. I also showed you how to test Functor laws with it in Javascript the last time. This time, we’ll see how to use it to test some real life work, taken (with a couple modifications) from one of our project’s code bases.

What the property

Let’s start by unwrapping that name. There are 2 important words in it:

  • Property ➡ It’s a fact. A predicate that will always hold true, regardless of what happens. I use it interchangeably with Law, but that’s probably not mathematically correct. Still, I use the analogy to remind myself that it’s something that prescribes accepted behaviour.
  • Testing ➡ A test is our way of assuring ourselves of something. Teachers test their students as a way to make sure that they learned what they should’ve learned. We test code to make sure it’s working as we intended it to work.

Honestly, I haven’t found a definition that says why there’s a Based in the middle, and why it’s not just Property Testing 🤷‍♂️.

So, Property Based Testing is a way for us to make sure our code always behaves in the same way, regardless of what happens. Now, for some code action!

Identifying Properties

We’re tasked with the following:

// Write a function that:
// Takes an array of strings
// Remove all instances of a dash character (-)
// Makes sure concatenating the strings will result in a string
// shorter than 200 characters
// (by removing all extra fluff from the tail)
// Makes all strings lowercase
// Returns an array of the resulting strings.

The use case is that we get a whole bunch of user generated strings that define flags on an api REST call. But the service can’t handle dashes or uppercase, for whatever reason, and anything beyond 200 characters will cause it to break. Why all this happens is out of our control, just bear with me 🐻.

So, there’s a couple o properties we can define here:

  • The resulting strings will never ever have a dash character.
  • A string made up of concatenating the result will have a length less than or equal to 200 characters.
  • There will be no instances of a result with uppercase characters on it.

So, the first step is to define these properties in code, like this:

// There will be no dashes ever
const noDashesProperty = xs => stringCleaning(xs).join("").indexOf("-") === -1;

// It will be shorter than 200 characters
const limitedStringLengthProperty = xs => stringCleaning(xs).join("").length <= 200;

// There will be no upper case characters in any string inside the result.
const lowerCaseProperty = xs => stringCleaning(xs).reduce((acc, cur) => acc && cur.toLowerCase() === cur, true);

We know the result will be an array of strings, so we can operate on that assumption.

Generating Input

We want to tell our testing framework (we’ll use jsverify again, so every mention of jsc are functions that come with it) what input can be passed to our function. We know we’ll always receive an array containing strings. Jsverify can do that for us like this:

// An array of random, arbitrary strings
jsc.array(jsc.string);

But one of our properties implies that we need some longer strings sometimes, just to make sure it’s behaving properly. So, we can tell jsverify to generate longer strings always:

// A non empty string that will have a length greater than 5.
const largeString = jsc.suchthat(jsc.nestring, str => str.length > 5);

Be careful with this though. If randomly generated values that pass the condition are scarce, this could take forever.

Writing the tests

I found that using jsverify with any of the usual test runner suspects is really straight forward, so I used Jest. Here’s what my tests look like:

describe("Our string manipulation function", () => {
  it("should remove all instances of dash (-)", () => {
    expect(jsc.checkForall(jsc.array(jsc.nestring), noDashesProperty)).toBe(true);
  });

  it("should make sure the resulting concatenation is shorter than 200 chars", () => {
    expect(jsc.checkForall(stringArrayArb, limitedStringLengthProperty)).toBe(true);
  });
  it("should make sure the resulting concatenation is shorter than 200 chars", () => {
    expect(jsc.checkForall(jsc.array(jsc.nestring), lowerCaseProperty)).toBe(true);
  });
});

The function jsc.checkForall takes two things:

  • It first takes any number of random generating values like jsc.nestring. That will be a random non-empty string.
  • The very last argument will be the function those values are passed to. This function must return a boolean value. In our case, we already had our property testing functions, so those are the ones we use.

jsc.checkForall will then return true if all of the function returns true for all of the test cases (100 by default). If any of them fail, it will return an object that contains the case that failed and why it failed. All that’s left, is writing our function to test.

Property driving

The first property says that we’ll return an array of strings and none of them will have a dash character. We can fulfil that one like this:

const stringCleaning = xs => xs.join(",").replace(/-/g, "").split(",");

The next property tells us that we should trim the results after the 200th character. Like:

const stringCleaning = xs => xs.join(",").replace(/-/g, "").substring(0, 200).split(",");

And finally, we need to make sure that there are no upper case characters in any string.

const stringCleaning = xs => xs.join(",").replace(/-/g, "").substring(0, 200).toLowerCase().split(",");

And that’s all! That’s our final function, fulfilling all of its properties and 100% driven by them. It’s important to notice here that the way Property Based Testing drives an implementation is a lot different than the way Unit Testing drives it. But it’s still possible, if we really want to.

I think Property Based Testing and Unit Testing are not mutually exclusive. I plan to drive implementations with Unit Tests and then add Property Tests when defining the properties is easy / useful, but not all the time. At the end of the day, it’s just one more tool in our tool belt. But it’s a powerful one if we use it wisely.

Here’s a Gist with the code: Property Based Testing String stuff · GitHub