Skip to content

Improve TLD lookup performance

minibikini requested to merge minibikini/auto_linker:improve-performance into master

Improves TLD lookup performance by changing TLDs list type from List to MapSet.

Benchmark Code:

tlds_list = "./priv/tlds.txt" |> File.read!() |> String.split("\n", trim: true)
tlds_set = MapSet.new(tlds_list)

tld = "tiffany"

Benchee.run(%{
  "list" => fn -> Enum.member?(tlds_list, tld) end,
  "map_set" => fn -> MapSet.member?(tlds_set, tld) end
})

Results:

Benchmark suite executing with the following configuration:
warmup: 2 s
time: 5 s
memory time: 0 ns
parallel: 1
inputs: none specified
Estimated total run time: 14 s

Benchmarking list...
Benchmarking map_set...

Name              ips        average  deviation         median         99th %
map_set        4.82 M        0.21 μs   ±268.31%           0 μs        0.98 μs
list         0.0903 M       11.08 μs    ±29.81%       10.98 μs       20.98 μs

Comparison:
map_set        4.82 M
list         0.0903 M - 53.39x slower +10.87 μs

Merge request reports