Adding Dialyzer without the Pain · The Phoenix Files

Why Check Types?

Erlang and Elixir are dynamically-typed, so why should we care about types? The nice part is that you aren’t strictly required to care about types at all. If you don’t want to write type definitions and check them with external tools, you don’t have to. However, many developers find some form of type specification useful as a form of documentation and a tool for preventing bugs. Luckily for us, Erlang and Elixir programmers are spoiled for choice when it comes to tooling that can help us through the use of types without needing to run our code first. The rest of this post will be an analysis of those tools as well as an example of how to implement a common Erlang/Elixir type-checking tool: Dialyzer.

The First Poll

A year ago, there was a poll posted to the Elixir Forum asking whether or not people used Dialyzer in their projects. Of the 207 votes (at the time of this post), about 72% of them used Dialyzer for at least some of their projects. The poll replies were where the real meat of the subject was; some highlights:

I find dialyzer to work well for projects, which are deployed. I tend to not like it for libraries, which don’t really use their code, because that usually means dialyzer doesn’t find half of the issues. - LostKobrakai
I have to admit that I do have quite a love/hate relationship with the Dialyzer. It does really catch potential errors, but the error messages can be really difficult to understand, especially when largish structs are involved. - xpg
No. Too slow, too cryptic. Hoping for something like Gradualizer to be usable. In the meantime, I invest time in end to end tests. - stefanchrobot

Admittedly, I’ve cherry-picked portions of responses that are critical of Dialyzer, but only to point out that Dialyzer is a controversial tool among Elixir developers.

The Second Poll

In order to get more opinions, I created a followup poll asking what tool (if any) people preferred to typecheck their Elixir/Erlang code. Poll options included:

As you can probably tell, we are fortunate to be spoiled for choice when it comes to type-checking in the Erlang/Elixir ecosystem, though of the 107 votes (at the time of this post), about 65% listed Dialyzer/Dialyzir as a typechecking tool they used with Elixir/Erlang. This seems pretty consistent with the previous poll. There weren’t as many replies as the original poll, but it still gave a good impression of the Elixir/Erlang type-checking landscape.

The Inspiration

Six months before making the followup poll, I watched a talk titled Slaying the Type Hydra, or How We Went from 12,000 Dialyzer Errors to None. In it, Jesper Eskilson describes how Klarna added Dialyzer into a preexisting system and how (at a high level) they integrated it into their CI and review process in order to steadily reduce their Dialyzer error count to zero. It was not an effortless/painless journey, but it was encouraging. I set out to try the same thing within my own company.

Adding Dialyzer to an Existing Project

As an exercise, let’s try to add Dialyzer (more specifically, Dialyxir) to some existing Elixir code. Remember that even though we may run mix dialyzer, we’re actually using Dialyxir as a convenience for Elixir projects. We’ll test out this process using a few large, well-known projects to see what we can learn:

Add The Dialyxir Dependency

This is the easy part:

# mix.exs
defp deps do
  [
    {:dialyxir, "~> 1.3", only: [:dev], runtime: false},
  ]
end

Followed by: mix do deps.get, deps.compile

Run Dialyzer

At this point, you’d normally run mix dialyzer, which would both generate your Dialyzer PLT files and then typecheck your project with them. We’re going to split up this step slightly for learning purposes.

Side Note: PLTs

Running the mix task dialyzer by default builds several PLT files:

NOTE: If you use the asdf language version manager, the location of $MIX_HOME will vary based on the current version of Elixir you are using: ~/.asdf/installs/elixir/$ELIXIR_VERSION/.mix

Generating PLTs

Running mix dialyzer --plt only generates the required PLT files to do typechecks. Time taken (all times are done on the same machine, a Macbook Pro with an M2 Pro):

Now we can run mix dialyzer to perform the actual typecheck and see how many errors we find!

Phoenix:

Total errors: 173, Skipped: 0, Unnecessary Skips: 0
done in 0m0.84s
# ...a bunch of errors printed here...
done (warnings were emitted)
Halting VM with exit status 2

Livebook

Total errors: 27, Skipped: 0, Unnecessary Skips: 0
done in 0m3.37s
# ...a bunch of errors printed here...
done (warnings were emitted)
Halting VM with exit status 2

Changelog:

Total errors: 74, Skipped: 0, Unnecessary Skips: 0
done in 0m3.2s
done (warnings were emitted)
Halting VM with exit status 2

Ignoring Existing Errors

In the previous step, you may have noticed the “skips” and “unnecessary skips” given when running Dialyzer. This is actually a special feature of dialyxir that allows us to selectively ignore certain errors using a special flag to mix dialyxir: --format ignore_file. So, let’s try it out for each of our projects using mix dialyzer --format ignore_file | wc -l to count how many lines are generated.

Preventing New Errors

With our new type-checking in place, let’s say someone makes a PR with an incorrect typespec on a new function in Phoenix:

# lib/phoenix.ex
# Here our `@spec` says the function's return type is an integer:
@spec my_cool_func(integer()) :: integer()
def my_cool_func(an_integer) do
  new_integer = an_integer + 1

# But the function is actually returning a string!
  Integer.to_string(new_integer)
end

Fixing Errors

Ignoring all our current errors is a great way to start using Dialyzer/Dialyxir, but eventually we’ll want to fix them. Let’s say we decide to ignore the above error by adding {"lib/phoenix.ex", :invalid_contract}, to the .dialyzer_ignore.exs file for the time being. Then we fix the error:

@spec my_cool_func(integer()) :: String.t()
def my_cool_func(an_integer) do
  new_integer = an_integer + 1
  Integer.to_string(new_integer)
end

Configuring CI

The next logical step is adding a Dialyzer check to our CI so that any PRs with new type errors are flagged and fixed before merging. There is a section of the Dialyxir docs that explains how to configure various CI tools to work well with Dialyxir.

Conclusion

We’ve seen how we can add Dialyzer late in the game to an Elixir project without getting overwhelmed. Tools like Dialyxir let us ignore all the legacy issues in our project today while helping us to keep it clean going forward. We can keep in mind that our goal is productivity, not perfection, and that it’s okay to ignore a type error so we can move forward today.