Skip to content
Commits on Source (186)
86 additional commits have been omitted to prevent performance issues.
Hide whitespace changes
Inline Side-by-side
.formatter.exs
View file @ 3f14b5b3
# Copyright © 2019-2022 Pleroma Authors
# SPDX-License-Identifier: MIT
# Used by "mix format"
[
inputs: ["{mix,.formatter}.exs", "{config,lib,test}/**/*.{ex,exs}"]
......
.gitignore
View file @ 3f14b5b3
# Copyright © 2017-2018 E-MetroTel
# Copyright © 2019-2022 Pleroma Authors
# SPDX-License-Identifier: MIT
# The directory Mix will write compiled artifacts to.
/_build
......
.gitlab-ci.yml
View file @ 3f14b5b3
image: elixir:1.7.2
# Copyright © 2019-2022 Pleroma Authors
# SPDX-License-Identifier: MIT
image: elixir:1.8.1
cache:
key: ${CI_COMMIT_REF_SLUG}
paths:
- deps
- _build
- deps
- _build
stages:
- lint
- test
- analysis
before_script:
- mix local.hex --force
......@@ -22,5 +26,11 @@ lint:
unit-testing:
stage: test
coverage: '/(\d+\.\d+\%) \| Total/'
script:
- mix test --trace --cover
analysis:
stage: analysis
script:
- mix test --trace
- mix credo --strict
.travis.yml
View file @ 3f14b5b3
# Copyright © 2017-2018 E-MetroTel
# SPDX-License-Identifier: MIT
language: elixir
elixir:
- 1.4.2
......
CHANGELOG.md 0 → 100644
View file @ 3f14b5b3
# Changelog
<!--
Copyright © 2019-2024 Pleroma Authors
SPDX-License-Identifier: MIT
-->
## 0.6.0 - 2024-??-??
### Changed
- No longer strips periods from the end of a link as it may break the link
### Fixed
- Incorrectly linked URLs where the domain part has a trailing character such as ,;:>?!
## 0.5.2 - 2022-01-09
### Fixed
- Fixed hashtags getting stripped at the end of lines.
## 0.5.1 - 2021-07-07
### Fixed
- Parsing crash with URLs ending in unbalanced closed paren, no path separator, and no query parameters
## 0.5.0 - 2021-03-02
### Added
- More robust detection of URLs inside a parenthetical
- Only link ip addresses with a scheme
- Fix mentions in markdown
- Fix mentions with apostrophe endings
## 0.4.1 - 2020-12-21
### Fixed
- Incorrect detection of IPv4 addresses causing random numbers (e.g., $123.45) to get linked
- Inability to link mentions with a trailing apostrophe. e.g., @user@example's
## 0.4.0 - 2020-11-24
### Added
- Support for linking URLs with FQDNs (e.g., "google.com.")
## 0.3.0 - 2020-11-17
### Added
- Support returning result as iodata and as safe iodata
### Fixed
- Hashtags followed by HTML tags "a", "code" and "pre" were not detected
- Incorrect parsing of HTML links inside HTML tags
- Punctuation marks in the end of urls were included in the html links
- Incorrect parsing of mentions with symbols before them
## 0.2.0 - 2020-07-21
### Added
- Added a `do_parse/4` clause to skip mentions when we're already skipping something else (eg, when inside a link)
### Fixed
- Fixed a typo in the readme
### Changed
- Refactored `Linkify.Parser.parse/2` to enumerate over the types instead of the opts
- Update dependencies
## 0.1.0 - 2019-07-11
- Initial release
LICENSE deleted 100644 → 0
View file @ 94193ca5
The MIT License (MIT)
Copyright (c) 2017 E-MetroTel
Permission is hereby granted, free of charge, to any person obtaining a copy
of this software and associated documentation files (the "Software"), to deal
in the Software without restriction, including without limitation the rights
to use, copy, modify, merge, publish, distribute, sublicense, and/or sell
copies of the Software, and to permit persons to whom the Software is
furnished to do so, subject to the following conditions:
The above copyright notice and this permission notice shall be included in all
copies or substantial portions of the Software.
THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR
IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY,
FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE
AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER
LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM,
OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE
SOFTWARE.
LICENSES/CC0-1.0.txt 0 → 100644
View file @ 3f14b5b3
Creative Commons Legal Code
CC0 1.0 Universal
CREATIVE COMMONS CORPORATION IS NOT A LAW FIRM AND DOES NOT PROVIDE
LEGAL SERVICES. DISTRIBUTION OF THIS DOCUMENT DOES NOT CREATE AN
ATTORNEY-CLIENT RELATIONSHIP. CREATIVE COMMONS PROVIDES THIS
INFORMATION ON AN "AS-IS" BASIS. CREATIVE COMMONS MAKES NO WARRANTIES
REGARDING THE USE OF THIS DOCUMENT OR THE INFORMATION OR WORKS
PROVIDED HEREUNDER, AND DISCLAIMS LIABILITY FOR DAMAGES RESULTING FROM
THE USE OF THIS DOCUMENT OR THE INFORMATION OR WORKS PROVIDED
HEREUNDER.
Statement of Purpose
The laws of most jurisdictions throughout the world automatically confer
exclusive Copyright and Related Rights (defined below) upon the creator
and subsequent owner(s) (each and all, an "owner") of an original work of
authorship and/or a database (each, a "Work").
Certain owners wish to permanently relinquish those rights to a Work for
the purpose of contributing to a commons of creative, cultural and
scientific works ("Commons") that the public can reliably and without fear
of later claims of infringement build upon, modify, incorporate in other
works, reuse and redistribute as freely as possible in any form whatsoever
and for any purposes, including without limitation commercial purposes.
These owners may contribute to the Commons to promote the ideal of a free
culture and the further production of creative, cultural and scientific
works, or to gain reputation or greater distribution for their Work in
part through the use and efforts of others.
For these and/or other purposes and motivations, and without any
expectation of additional consideration or compensation, the person
associating CC0 with a Work (the "Affirmer"), to the extent that he or she
is an owner of Copyright and Related Rights in the Work, voluntarily
elects to apply CC0 to the Work and publicly distribute the Work under its
terms, with knowledge of his or her Copyright and Related Rights in the
Work and the meaning and intended legal effect of CC0 on those rights.
1. Copyright and Related Rights. A Work made available under CC0 may be
protected by copyright and related or neighboring rights ("Copyright and
Related Rights"). Copyright and Related Rights include, but are not
limited to, the following:
i. the right to reproduce, adapt, distribute, perform, display,
communicate, and translate a Work;
ii. moral rights retained by the original author(s) and/or performer(s);
iii. publicity and privacy rights pertaining to a person's image or
likeness depicted in a Work;
iv. rights protecting against unfair competition in regards to a Work,
subject to the limitations in paragraph 4(a), below;
v. rights protecting the extraction, dissemination, use and reuse of data
in a Work;
vi. database rights (such as those arising under Directive 96/9/EC of the
European Parliament and of the Council of 11 March 1996 on the legal
protection of databases, and under any national implementation
thereof, including any amended or successor version of such
directive); and
vii. other similar, equivalent or corresponding rights throughout the
world based on applicable law or treaty, and any national
implementations thereof.
2. Waiver. To the greatest extent permitted by, but not in contravention
of, applicable law, Affirmer hereby overtly, fully, permanently,
irrevocably and unconditionally waives, abandons, and surrenders all of
Affirmer's Copyright and Related Rights and associated claims and causes
of action, whether now known or unknown (including existing as well as
future claims and causes of action), in the Work (i) in all territories
worldwide, (ii) for the maximum duration provided by applicable law or
treaty (including future time extensions), (iii) in any current or future
medium and for any number of copies, and (iv) for any purpose whatsoever,
including without limitation commercial, advertising or promotional
purposes (the "Waiver"). Affirmer makes the Waiver for the benefit of each
member of the public at large and to the detriment of Affirmer's heirs and
successors, fully intending that such Waiver shall not be subject to
revocation, rescission, cancellation, termination, or any other legal or
equitable action to disrupt the quiet enjoyment of the Work by the public
as contemplated by Affirmer's express Statement of Purpose.
3. Public License Fallback. Should any part of the Waiver for any reason
be judged legally invalid or ineffective under applicable law, then the
Waiver shall be preserved to the maximum extent permitted taking into
account Affirmer's express Statement of Purpose. In addition, to the
extent the Waiver is so judged Affirmer hereby grants to each affected
person a royalty-free, non transferable, non sublicensable, non exclusive,
irrevocable and unconditional license to exercise Affirmer's Copyright and
Related Rights in the Work (i) in all territories worldwide, (ii) for the
maximum duration provided by applicable law or treaty (including future
time extensions), (iii) in any current or future medium and for any number
of copies, and (iv) for any purpose whatsoever, including without
limitation commercial, advertising or promotional purposes (the
"License"). The License shall be deemed effective as of the date CC0 was
applied by Affirmer to the Work. Should any part of the License for any
reason be judged legally invalid or ineffective under applicable law, such
partial invalidity or ineffectiveness shall not invalidate the remainder
of the License, and in such case Affirmer hereby affirms that he or she
will not (i) exercise any of his or her remaining Copyright and Related
Rights in the Work or (ii) assert any associated claims and causes of
action with respect to the Work, in either case contrary to Affirmer's
express Statement of Purpose.
4. Limitations and Disclaimers.
a. No trademark or patent rights held by Affirmer are waived, abandoned,
surrendered, licensed or otherwise affected by this document.
b. Affirmer offers the Work as-is and makes no representations or
warranties of any kind concerning the Work, express, implied,
statutory or otherwise, including without limitation warranties of
title, merchantability, fitness for a particular purpose, non
infringement, or the absence of latent or other defects, accuracy, or
the present or absence of errors, whether or not discoverable, all to
the greatest extent permissible under applicable law.
c. Affirmer disclaims responsibility for clearing rights of other persons
that may apply to the Work or any use thereof, including without
limitation any person's Copyright and Related Rights in the Work.
Further, Affirmer disclaims responsibility for obtaining any necessary
consents, permissions or other rights required for any use of the
Work.
d. Affirmer understands and acknowledges that Creative Commons is not a
party to this document and has no duty or obligation with respect to
this CC0 or use of the Work.
LICENSES/MIT.txt 0 → 100644
View file @ 3f14b5b3
MIT License
Copyright (c) <year> <copyright holders>
Permission is hereby granted, free of charge, to any person obtaining a copy of this software and associated documentation files (the "Software"), to deal in the Software without restriction, including without limitation the rights to use, copy, modify, merge, publish, distribute, sublicense, and/or sell copies of the Software, and to permit persons to whom the Software is furnished to do so, subject to the following conditions:
The above copyright notice and this permission notice shall be included in all copies or substantial portions of the Software.
THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY, FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM, OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE SOFTWARE.
README.md
View file @ 3f14b5b3
# AutoLinker
# Linkify
[![Build Status](https://travis-ci.org/smpallen99/auto_linker.png?branch=master)](https://travis-ci.org/smpallen99/auto_linker) [![Hex Version][hex-img]][hex] [![License][license-img]][license]
[hex-img]: https://img.shields.io/hexpm/v/auto_linker.svg
[hex]: https://hex.pm/packages/auto_linker
[license-img]: http://img.shields.io/badge/license-MIT-brightgreen.svg
[license]: http://opensource.org/licenses/MIT
AutoLinker is a basic package for turning website names, and phone numbers into links.
Linkify is a basic package for turning website names into links.
Use this package in your web view to convert web references into click-able links.
This is a very early version. Some of the described options are not yet functional.
## Installation
The package can be installed by adding `auto_linker` to your list of dependencies in `mix.exs`:
The package can be installed by adding `linkify` to your list of dependencies in `mix.exs`:
```elixir
def deps do
[{:auto_linker, "~> 0.2"}]
[{:linkify, "~> 0.5"}]
end
```
......@@ -27,45 +18,36 @@ end
The following examples illustrate some examples on how to use the auto linker.
```iex
iex> AutoLinker.link("google.com")
"<a href='http://google.com' class='auto-linker' target='_blank' rel='noopener noreferrer'>google.com</a>"
iex> AutoLinker.link("google.com", new_window: false, rel: false)
"<a href='http://google.com' class='auto-linker'>google.com</a>"
```elixir
iex> AutoLinker.link("google.com", new_window: false, rel: false, class: false)
"<a href='http://google.com'>google.com</a>"
iex> Linkify.link("google.com")
"<a href=\"http://google.com\">google.com</a>"
iex> AutoLinker.link("call me at x9999", phone: true)
"call me at <a href=\"#\" class=\"phone-number\" data-phone=\"9999\">x9999</a>"
iex> Linkify.link("google.com", class: "linkified")
"<a href=\"http://google.com\" class=\"linkified\">google.com</a>"
iex> AutoLinker.link("or at home on 555.555.5555", phone: true)
"or at home on <a href=\"#\" class=\"phone-number\" data-phone=\"5555555555\">555.555.5555</a>"
iex> Linkify.link("google.com", new_window: true)
"<a href=\"http://google.com\" target=\"_blank\">google.com</a>"
iex> AutoLinker.link(", work (555) 555-5555", phone: true)
", work <a href=\"#\" class=\"phone-number\" data-phone=\"5555555555\">(555) 555-5555</a>"
iex> Linkify.link("google.com", new_window: true, rel: "noopener noreferrer")
"<a href=\"http://google.com\" target=\"_blank\" rel=\"noopener noreferrer\">google.com</a>"
iex> AutoLinker.link("[Google Search](http://google.com)", markdown: true)
"<a href='http://google.com' class='auto-linker' target='_blank' rel='noopener noreferrer'>Google Search</a>"
iex> Linkify.link("Hello @niceguy17@pleroma.com", mention: true, mention_prefix: "/users/")
"Hello <a href=\"/users/niceguy17@pleroma.com\">@niceguy17@pleroma.com</a>"
```
See the [Docs](https://hexdocs.pm/auto_linker/) for more examples
See the [Docs](https://hexdocs.pm/linkify/) for more examples
## Configuration
By default, link parsing is enabled and phone parsing is disabled.
```elixir
# enable phone parsing, and disable link parsing
config :auto_linker, opts: [phone: true, url: false]
```
## Acknowledgments
This is a fork of [auto_linker](https://github.com/smpallen99/auto_linker) by [Steve Pallen](https://github.com/smpallen99).
## License
`auto_linker` is Copyright (c) 2017 E-MetroTel
Copyright © 2017-2018 E-MetroTel
Copyright © 2019-2022 Pleroma Authors
The source is released under the MIT License.
SPDX-License-Identifier: MIT AND CC0-1.0
Check [LICENSE](LICENSE) for more information.
Check [REUSE Specification](https://reuse.software/spec/) on how to get more information.
config/config.exs deleted 100644 → 0
View file @ 94193ca5
# This file is responsible for configuring your application
# and its dependencies with the aid of the Mix.Config module.
use Mix.Config
# This configuration is loaded before any dependency and is restricted
# to this project. If another project depends on this project, this
# file won't be loaded nor affect the parent project. For this reason,
# if you want to provide default values for your application for
# 3rd-party users, it should be done in your "mix.exs" file.
# You can configure for your application as:
#
# config :auto_linker, key: :value
#
# And access this configuration in your application as:
#
# Application.get_env(:auto_linker, :key)
#
# Or configure a 3rd-party app:
#
# config :logger, level: :info
#
# It is also possible to import configuration files, relative to this
# directory. For example, you can emulate configuration per environment
# by uncommenting the line below and defining dev.exs, test.exs and such.
# Configuration from the imported file will override the ones defined
# here (which is why it is important to import them last).
#
# import_config "#{Mix.env}.exs"
lib/auto_linker.ex deleted 100644 → 0
View file @ 94193ca5
defmodule AutoLinker do
@moduledoc """
Create url links from text containing urls.
Turns an input string like `"Check out google.com"` into
`Check out "<a href=\"http://google.com\" target=\"_blank\" rel=\"noopener noreferrer\">google.com</a>"`
## Examples
iex> AutoLinker.link("google.com")
~s(<a href="http://google.com" class="auto-linker" target="_blank" rel="noopener noreferrer">google.com</a>)
iex> AutoLinker.link("google.com", new_window: false, rel: false)
~s(<a href="http://google.com" class="auto-linker">google.com</a>)
iex> AutoLinker.link("google.com", new_window: false, rel: false, class: false)
~s(<a href="http://google.com">google.com</a>)
iex> AutoLinker.link("[Google](http://google.com)", markdown: true, new_window: false, rel: false, class: false)
~s(<a href='http://google.com'>Google</a>)
iex> AutoLinker.link("[Google Search](http://google.com)", markdown: true)
~s(<a href='http://google.com' class="auto-linker" target="_blank" rel="noopener noreferrer">Google Search</a>)
"""
import AutoLinker.Parser
@doc """
Auto link a string.
Options:
* `class: "auto-linker"` - specify the class to be added to the generated link. false to clear
* `rel: "noopener noreferrer"` - override the rel attribute. false to clear
* `new_window: true` - set to false to remove `target='_blank'` attribute
* `scheme: false` - Set to true to link urls with schema `http://google`
* `truncate: false` - Set to a number to truncate urls longer then the number. Truncated urls will end in `..`
* `strip_prefix: true` - Strip the scheme prefix
* `exclude_class: false` - Set to a class name when you don't want urls auto linked in the html of the give class
* `exclude_id: false` - Set to an element id when you don't want urls auto linked in the html of the give element
* `exclude_patterns: ["```"]` - Don't link anything between the the pattern
* `markdown: false` - link markdown style links
* `email: false` - link email links
* `mention: false` - link @mentions (when `true`, requires `mention_prefix` or `mention_handler` options to be set)
* `mention_prefix: nil` - a prefix to build a link for a mention (example: `https://example.com/user/`)
* `mention_handler: nil` - a custom handler to validate and formart a mention
* `hashtag: false` - link #hashtags (when `true`, requires `hashtag_prefix` or `hashtag_handler` options to be set)
* `hashtag_prefix: nil` - a prefix to build a link for a hashtag (example: `https://example.com/tag/`)
* `hashtag_handler: nil` - a custom handler to validate and formart a hashtag
* `extra: false` - link urls with rarely used schemes (magnet, ipfs, irc, etc.)
Each of the above options can be specified when calling `link(text, opts)`
or can be set in the `:auto_linker`'s configuration. For example:
config :auto_linker,
class: false,
new_window: false
Note that passing opts to `link/2` will override the configuration settings.
"""
def link(text, opts \\ []) do
parse(text, opts)
end
def link_map(text, acc, opts \\ []) do
parse({text, acc}, opts)
end
end
lib/auto_linker/parser.ex deleted 100644 → 0
View file @ 94193ca5
defmodule AutoLinker.Parser do
@moduledoc """
Module to handle parsing the the input string.
"""
alias AutoLinker.Builder
@doc """
Parse the given string, identifying items to link.
Parses the string, replacing the matching urls and phone numbers with an html link.
## Examples
iex> AutoLinker.Parser.parse("Check out google.com")
~s{Check out <a href="http://google.com" class="auto-linker" target="_blank" rel="noopener noreferrer">google.com</a>}
iex> AutoLinker.Parser.parse("call me at x9999", phone: true)
~s{call me at <a href="#" class="phone-number" data-phone="9999">x9999</a>}
iex> AutoLinker.Parser.parse("or at home on 555.555.5555", phone: true)
~s{or at home on <a href="#" class="phone-number" data-phone="5555555555">555.555.5555</a>}
iex> AutoLinker.Parser.parse(", work (555) 555-5555", phone: true)
~s{, work <a href="#" class="phone-number" data-phone="5555555555">(555) 555-5555</a>}
"""
@invalid_url ~r/(\.\.+)|(^(\d+\.){1,2}\d+$)/
@match_url ~r{^[\w\.-]+(?:\.[\w\.-]+)+[\w\-\._~%:/?#[\]@!\$&'\(\)\*\+,;=.]+$}
@match_scheme ~r{^(?:http(s)?:\/\/)?[\w.-]+(?:\.[\w\.-]+)+[\w\-\._~%:/?#[\]@!\$&'\(\)\*\+,;=.]+$}
@match_phone ~r"((?:x\d{2,7})|(?:(?:\+?1\s?(?:[.-]\s?)?)?(?:\(\s?(?:[2-9]1[02-9]|[2-9][02-8]1|[2-9][02-8][02-9])\s?\)|(?:[2-9]1[02-9]|[2-9][02-8]1|[2-9][02-8][02-9]))\s?(?:[.-]\s?)?)(?:[2-9]1[02-9]|[2-9][02-9]1|[2-9][02-9]{2})\s?(?:[.-]\s?)?(?:[0-9]{4}))"
@match_hostname ~r{^(?:https?:\/\/)?(?:[^@\n]+\\w@)?(?<host>[^:#~\/\n?]+)}
@match_ip ~r"^(([0-9]|[1-9][0-9]|1[0-9]{2}|2[0-4][0-9]|25[0-5])\.){3}([0-9]|[1-9][0-9]|1[0-9]{2}|2[0-4][0-9]|25[0-5])$"
# @user
# @user@example.com
@match_mention ~r/^@[a-zA-Z\d_-]+@[a-zA-Z0-9_-](?:[a-zA-Z0-9-]{0,61}[a-zA-Z0-9])?(?:\.[a-zA-Z0-9](?:[a-zA-Z0-9-]{0,61}[a-zA-Z0-9])?)*|@[a-zA-Z\d_-]+/u
# https://www.w3.org/TR/html5/forms.html#valid-e-mail-address
@match_email ~r/^[a-zA-Z0-9.!#$%&'*+\/=?^_`{|}~-]+@[a-zA-Z0-9](?:[a-zA-Z0-9-]{0,61}[a-zA-Z0-9])?(?:\.[a-zA-Z0-9](?:[a-zA-Z0-9-]{0,61}[a-zA-Z0-9])?)*$/u
@match_hashtag ~r/^(?<tag>\#[[:word:]_]*[[:alpha:]_·][[:word:]_·]*)/u
@prefix_extra [
"magnet:?",
"dweb://",
"dat://",
"gopher://",
"ipfs://",
"ipns://",
"irc://",
"ircs://",
"irc6://",
"mumble://",
"ssb://"
]
@tlds "./priv/tlds.txt" |> File.read!() |> String.split("\n", trim: true)
@default_opts ~w(url)a
def parse(input, opts \\ %{})
def parse(input, opts) when is_binary(input), do: {input, nil} |> parse(opts) |> elem(0)
def parse(input, list) when is_list(list), do: parse(input, Enum.into(list, %{}))
def parse(input, opts) do
config =
:auto_linker
|> Application.get_env(:opts, [])
|> Enum.into(%{})
|> Map.put(
:attributes,
Application.get_env(:auto_linker, :attributes, [])
)
opts =
Enum.reduce(@default_opts, opts, fn opt, acc ->
if is_nil(opts[opt]) and is_nil(config[opt]) do
Map.put(acc, opt, true)
else
acc
end
end)
do_parse(input, Map.merge(config, opts))
end
defp do_parse(input, %{phone: false} = opts), do: do_parse(input, Map.delete(opts, :phone))
defp do_parse(input, %{url: false} = opts), do: do_parse(input, Map.delete(opts, :url))
defp do_parse(input, %{phone: _} = opts) do
input
|> do_parse(opts, {"", "", :parsing}, &check_and_link_phone/3)
|> do_parse(Map.delete(opts, :phone))
end
defp do_parse(input, %{hashtag: true} = opts) do
input
|> do_parse(opts, {"", "", :parsing}, &check_and_link_hashtag/3)
|> do_parse(Map.delete(opts, :hashtag))
end
defp do_parse(input, %{extra: true} = opts) do
input
|> do_parse(opts, {"", "", :parsing}, &check_and_link_extra/3)
|> do_parse(Map.delete(opts, :extra))
end
defp do_parse({text, user_acc}, %{markdown: true} = opts) do
text
|> Builder.create_markdown_links(opts)
|> (&{&1, user_acc}).()
|> do_parse(Map.delete(opts, :markdown))
end
defp do_parse(input, %{email: true} = opts) do
input
|> do_parse(opts, {"", "", :parsing}, &check_and_link_email/3)
|> do_parse(Map.delete(opts, :email))
end
defp do_parse({text, user_acc}, %{url: _} = opts) do
input =
with exclude <- Map.get(opts, :exclude_patterns),
true <- is_list(exclude),
true <- String.starts_with?(text, exclude) do
{text, user_acc}
else
_ ->
do_parse(
{text, user_acc},
opts,
{"", "", :parsing},
&check_and_link/3
)
end
do_parse(input, Map.delete(opts, :url))
end
defp do_parse(input, %{mention: true} = opts) do
input
|> do_parse(opts, {"", "", :parsing}, &check_and_link_mention/3)
|> do_parse(Map.delete(opts, :mention))
end
defp do_parse(input, _), do: input
defp do_parse({"", user_acc}, _opts, {"", acc, _}, _handler),
do: {acc, user_acc}
defp do_parse({"", user_acc}, opts, {buffer, acc, _}, handler) do
{buffer, user_acc} = run_handler(handler, buffer, opts, user_acc)
{acc <> buffer, user_acc}
end
defp do_parse({"<a" <> text, user_acc}, opts, {buffer, acc, :parsing}, handler),
do: do_parse({text, user_acc}, opts, {"", acc <> buffer <> "<a", :skip}, handler)
defp do_parse({"</a>" <> text, user_acc}, opts, {buffer, acc, :skip}, handler),
do: do_parse({text, user_acc}, opts, {"", acc <> buffer <> "</a>", :parsing}, handler)
defp do_parse({"<" <> text, user_acc}, opts, {"", acc, :parsing}, handler),
do: do_parse({text, user_acc}, opts, {"<", acc, {:open, 1}}, handler)
defp do_parse({">" <> text, user_acc}, opts, {buffer, acc, {:attrs, level}}, handler),
do:
do_parse(
{text, user_acc},
opts,
{"", acc <> buffer <> ">", {:html, level}},
handler
)
defp do_parse({<<ch::8>> <> text, user_acc}, opts, {"", acc, {:attrs, level}}, handler) do
do_parse({text, user_acc}, opts, {"", acc <> <<ch::8>>, {:attrs, level}}, handler)
end
defp do_parse({"</" <> text, user_acc}, opts, {buffer, acc, {:html, level}}, handler) do
{buffer, user_acc} = run_handler(handler, buffer, opts, user_acc)
do_parse(
{text, user_acc},
opts,
{"", acc <> buffer <> "</", {:close, level}},
handler
)
end
defp do_parse({">" <> text, user_acc}, opts, {buffer, acc, {:close, 1}}, handler),
do: do_parse({text, user_acc}, opts, {"", acc <> buffer <> ">", :parsing}, handler)
defp do_parse({">" <> text, user_acc}, opts, {buffer, acc, {:close, level}}, handler),
do:
do_parse(
{text, user_acc},
opts,
{"", acc <> buffer <> ">", {:html, level - 1}},
handler
)
defp do_parse(
{<<char::bytes-size(1), text::binary>>, user_acc},
opts,
{buffer, acc, {:open, level}},
handler
)
when char in [" ", "\r", "\n"] do
do_parse(
{text, user_acc},
opts,
{"", acc <> buffer <> char, {:attrs, level}},
handler
)
end
# default cases where state is not important
defp do_parse(
{" " <> text, user_acc},
%{phone: _} = opts,
{buffer, acc, state},
handler
),
do: do_parse({text, user_acc}, opts, {buffer <> " ", acc, state}, handler)
defp do_parse(
{<<char::bytes-size(1), text::binary>>, user_acc},
opts,
{buffer, acc, state},
handler
)
when char in [" ", "\r", "\n"] do
{buffer, user_acc} = run_handler(handler, buffer, opts, user_acc)
do_parse(
{text, user_acc},
opts,
{"", acc <> buffer <> char, state},
handler
)
end
defp do_parse({<<ch::8>>, user_acc}, opts, {buffer, acc, state}, handler) do
{buffer, user_acc} = run_handler(handler, buffer <> <<ch::8>>, opts, user_acc)
do_parse(
{"", user_acc},
opts,
{"", acc <> buffer, state},
handler
)
end
defp do_parse({<<ch::8>> <> text, user_acc}, opts, {buffer, acc, state}, handler),
do: do_parse({text, user_acc}, opts, {buffer <> <<ch::8>>, acc, state}, handler)
def check_and_link(buffer, opts, _user_acc) do
buffer
|> is_url?(opts[:scheme])
|> link_url(buffer, opts)
end
def check_and_link_email(buffer, opts, _user_acc) do
buffer
|> is_email?
|> link_email(buffer, opts)
end
def check_and_link_phone(buffer, opts, _user_acc) do
buffer
|> match_phone
|> link_phone(buffer, opts)
end
def check_and_link_mention(buffer, opts, user_acc) do
buffer
|> match_mention
|> link_mention(buffer, opts, user_acc)
end
def check_and_link_hashtag(buffer, opts, user_acc) do
buffer
|> match_hashtag
|> link_hashtag(buffer, opts, user_acc)
end
def check_and_link_extra("xmpp:" <> handle, opts, _user_acc) do
handle
|> is_email?
|> link_extra("xmpp:" <> handle, opts)
end
def check_and_link_extra(buffer, opts, _user_acc) do
buffer
|> String.starts_with?(@prefix_extra)
|> link_extra(buffer, opts)
end
# @doc false
def is_url?(buffer, true) do
if Regex.match?(@invalid_url, buffer) do
false
else
@match_scheme |> Regex.match?(buffer) |> is_valid_tld?(buffer)
end
end
def is_url?(buffer, _) do
if Regex.match?(@invalid_url, buffer) do
false
else
@match_url |> Regex.match?(buffer) |> is_valid_tld?(buffer)
end
end
def is_email?(buffer) do
if Regex.match?(@invalid_url, buffer) do
false
else
@match_email |> Regex.match?(buffer) |> is_valid_tld?(buffer)
end
end
def is_valid_tld?(true, buffer) do
[host] = Regex.run(@match_hostname, buffer, capture: [:host])
if is_ip?(host) do
true
else
tld = host |> String.split(".") |> List.last()
Enum.member?(@tlds, tld)
end
end
def is_valid_tld?(false, _), do: false
def is_ip?(buffer) do
Regex.match?(@match_ip, buffer)
end
@doc false
def match_phone(buffer) do
case Regex.scan(@match_phone, buffer) do
[] -> nil
other -> other
end
end
def match_mention(buffer) do
case Regex.run(@match_mention, buffer) do
[mention] -> mention
_ -> nil
end
end
def match_hashtag(buffer) do
case Regex.run(@match_hashtag, buffer, capture: [:tag]) do
[hashtag] -> hashtag
_ -> nil
end
end
def link_hashtag(nil, buffer, _, _user_acc), do: buffer
def link_hashtag(hashtag, buffer, %{hashtag_handler: hashtag_handler} = opts, user_acc) do
hashtag
|> hashtag_handler.(buffer, opts, user_acc)
|> maybe_update_buffer(hashtag, buffer)
end
def link_hashtag(hashtag, buffer, opts, _user_acc) do
hashtag
|> Builder.create_hashtag_link(buffer, opts)
|> maybe_update_buffer(hashtag, buffer)
end
def link_mention(nil, buffer, _, user_acc), do: {buffer, user_acc}
def link_mention(mention, buffer, %{mention_handler: mention_handler} = opts, user_acc) do
mention
|> mention_handler.(buffer, opts, user_acc)
|> maybe_update_buffer(mention, buffer)
end
def link_mention(mention, buffer, opts, _user_acc) do
mention
|> Builder.create_mention_link(buffer, opts)
|> maybe_update_buffer(mention, buffer)
end
defp maybe_update_buffer(out, match, buffer) when is_binary(out) do
maybe_update_buffer({out, nil}, match, buffer)
end
defp maybe_update_buffer({out, user_acc}, match, buffer)
when match != buffer and out != buffer do
out = String.replace(buffer, match, out)
{out, user_acc}
end
defp maybe_update_buffer(out, _match, _buffer), do: out
def link_phone(nil, buffer, _), do: buffer
def link_phone(list, buffer, opts) do
Builder.create_phone_link(list, buffer, opts)
end
@doc false
def link_url(true, buffer, opts) do
Builder.create_link(buffer, opts)
end
def link_url(_, buffer, _opts), do: buffer
@doc false
def link_email(true, buffer, opts) do
Builder.create_email_link(buffer, opts)
end
def link_email(_, buffer, _opts), do: buffer
def link_extra(true, buffer, opts) do
Builder.create_extra_link(buffer, opts)
end
def link_extra(_, buffer, _opts), do: buffer
defp run_handler(handler, buffer, opts, user_acc) do
case handler.(buffer, opts, user_acc) do
{buffer, user_acc} -> {buffer, user_acc}
buffer -> {buffer, user_acc}
end
end
end
lib/linkify.ex 0 → 100644
View file @ 3f14b5b3
# Copyright © 2017-2018 E-MetroTel
# Copyright © 2019-2022 Pleroma Authors
# SPDX-License-Identifier: MIT
defmodule Linkify do
@moduledoc """
Create url links from text containing urls.
Turns an input string like `"Check out google.com"` into
`Check out "<a href=\"http://google.com\">google.com</a>"`
## Examples
iex> Linkify.link("google.com")
~s(<a href="http://google.com">google.com</a>)
iex> Linkify.link("google.com", new_window: true, rel: "noopener noreferrer")
~s(<a href="http://google.com" target="_blank" rel="noopener noreferrer">google.com</a>)
iex> Linkify.link("google.com", class: "linkified")
~s(<a href="http://google.com" class="linkified">google.com</a>)
"""
import Linkify.Parser
@doc """
Finds links and turns them into HTML `<a>` tag.
Options:
* `class` - specify the class to be added to the generated link.
* `rel` - specify the rel attribute.
* `new_window` - set to `true` to add `target="_blank"` attribute
* `truncate` - Set to a number to truncate urls longer then the number. Truncated urls will end in `...`
* `strip_prefix` - Strip the scheme prefix (default: `false`)
* `exclude_class` - Set to a class name when you don't want urls auto linked in the html of the give class (default: `false`)
* `exclude_id` - Set to an element id when you don't want urls auto linked in the html of the give element (default: `false`)
* `email` - link email links (default: `false`)
* `mention` - link @mentions (when `true`, requires `mention_prefix` or `mention_handler` options to be set) (default: `false`)
* `mention_prefix` - a prefix to build a link for a mention (example: `https://example.com/user/`, default: `nil`)
* `mention_handler` - a custom handler to validate and format a mention (default: `nil`)
* `hashtag: false` - link #hashtags (when `true`, requires `hashtag_prefix` or `hashtag_handler` options to be set)
* `hashtag_prefix: nil` - a prefix to build a link for a hashtag (example: `https://example.com/tag/`)
* `hashtag_handler: nil` - a custom handler to validate and format a hashtag
* `extra: false` - link urls with rarely used schemes (magnet, ipfs, irc, etc.)
* `validate_tld: true` - Set to false to disable TLD validation for urls/emails, also can be set to :no_scheme to validate TLDs only for urls without a scheme (e.g `example.com` will be validated, but `http://example.loki` won't)
* `iodata` - Set to `true` to return iodata as a result, or `:safe` for iodata with linkified anchor tags wrapped in Phoenix.HTML `:safe` tuples (removes need for further sanitization)
* `href_handler: nil` - a custom handler to process a url before it is set as the link href, useful for generating exit links
"""
def link(text, opts \\ []) do
parse(text, opts)
end
def link_to_iodata(text, opts \\ []) do
parse(text, Keyword.merge(opts, iodata: true))
end
def link_safe(text, opts \\ []) do
parse(text, Keyword.merge(opts, iodata: :safe))
end
def link_map(text, acc, opts \\ []) do
parse({text, acc}, opts)
end
end
lib/auto_linker/builder.ex → lib/linkify/builder.ex
View file @ 3f14b5b3
defmodule AutoLinker.Builder do
# Copyright © 2017-2018 E-MetroTel
# Copyright © 2019-2022 Pleroma Authors
# SPDX-License-Identifier: MIT
defmodule Linkify.Builder do
@moduledoc """
Module for building the auto generated link.
"""
......@@ -17,14 +21,6 @@ defmodule AutoLinker.Builder do
|> format_url(text, opts)
end
def create_markdown_links(text, opts) do
[]
|> build_attrs(text, opts, :rel)
|> build_attrs(text, opts, :target)
|> build_attrs(text, opts, :class)
|> format_markdown(text, opts)
end
defp build_attrs(attrs, uri, %{rel: get_rel}, :rel) when is_function(get_rel, 1) do
case get_rel.(uri) do
nil -> attrs
......@@ -33,19 +29,28 @@ defmodule AutoLinker.Builder do
end
defp build_attrs(attrs, _, opts, :rel) do
if rel = Map.get(opts, :rel, "noopener noreferrer"), do: [{:rel, rel} | attrs], else: attrs
case Map.get(opts, :rel) do
rel when is_binary(rel) -> [{:rel, rel} | attrs]
_ -> attrs
end
end
defp build_attrs(attrs, _, opts, :target) do
if Map.get(opts, :new_window, true), do: [{:target, :_blank} | attrs], else: attrs
if Map.get(opts, :new_window), do: [{:target, :_blank} | attrs], else: attrs
end
defp build_attrs(attrs, _, opts, :class) do
if cls = Map.get(opts, :class, "auto-linker"), do: [{:class, cls} | attrs], else: attrs
case Map.get(opts, :class) do
cls when is_binary(cls) -> [{:class, cls} | attrs]
_ -> attrs
end
end
defp build_attrs(attrs, url, _opts, :href) do
[{:href, url} | attrs]
defp build_attrs(attrs, url, opts, :href) do
case Map.get(opts, :href_handler) do
handler when is_function(handler) -> [{:href, handler.(url)} | attrs]
_ -> [{:href, url} | attrs]
end
end
defp add_scheme("http://" <> _ = url), do: url
......@@ -55,34 +60,24 @@ defmodule AutoLinker.Builder do
defp format_url(attrs, url, opts) do
url =
url
|> strip_prefix(Map.get(opts, :strip_prefix, true))
|> strip_prefix(Map.get(opts, :strip_prefix, false))
|> truncate(Map.get(opts, :truncate, false))
attrs = format_attrs(attrs)
"<a #{attrs}>#{url}</a>"
attrs
|> format_attrs()
|> format_tag(url, opts)
end
defp format_attrs(attrs) do
attrs
|> Enum.map(fn {key, value} -> ~s(#{key}="#{value}") end)
|> Enum.join(" ")
end
defp format_markdown(attrs, text, _opts) do
attrs =
case format_attrs(attrs) do
"" -> ""
attrs -> " " <> attrs
end
Regex.replace(~r/\[(.+?)\]\((.+?)\)/, text, "<a href='\\2'#{attrs}>\\1</a>")
|> Enum.map_join(" ", fn {key, value} -> ~s(#{key}="#{value}") end)
end
defp truncate(url, false), do: url
defp truncate(url, len) when len < 3, do: url
defp truncate(url, len) do
if String.length(url) > len, do: String.slice(url, 0, len - 2) <> "..", else: url
if String.length(url) > len, do: String.slice(url, 0, len - 2) <> "...", else: url
end
defp strip_prefix(url, true) do
......@@ -93,34 +88,6 @@ defmodule AutoLinker.Builder do
defp strip_prefix(url, _), do: url
def create_phone_link([], buffer, _), do: buffer
def create_phone_link([h | t], buffer, opts) do
create_phone_link(t, format_phone_link(h, buffer, opts), opts)
end
def format_phone_link([h | _], buffer, opts) do
val =
h
|> String.replace(~r/[\.\+\- x\(\)]+/, "")
|> format_phone_link(h, opts)
# val = ~s'<a href="#" class="phone-number" data-phone="#{number}">#{h}</a>'
String.replace(buffer, h, val)
end
def format_phone_link(number, original, opts) do
tag = opts[:tag] || "a"
class = opts[:class] || "phone-number"
data_phone = opts[:data_phone] || "data-phone"
attrs = format_attributes(opts[:attributes] || [])
href = opts[:href] || "#"
~s'<#{tag} href="#{href}" class="#{class}" #{data_phone}="#{number}"#{attrs}>#{original}</#{
tag
}>'
end
def create_mention_link("@" <> name, _buffer, opts) do
mention_prefix = opts[:mention_prefix]
......@@ -163,29 +130,39 @@ defmodule AutoLinker.Builder do
|> format_extra(uri, opts)
end
def format_mention(attrs, name, _opts) do
attrs = format_attrs(attrs)
"<a #{attrs}>@#{name}</a>"
def format_mention(attrs, name, opts) do
attrs
|> format_attrs()
|> format_tag("@#{name}", opts)
end
def format_hashtag(attrs, tag, opts) do
attrs
|> format_attrs()
|> format_tag("##{tag}", opts)
end
def format_hashtag(attrs, tag, _opts) do
attrs = format_attrs(attrs)
"<a #{attrs}>##{tag}</a>"
def format_email(attrs, email, opts) do
attrs
|> format_attrs()
|> format_tag(email, opts)
end
def format_extra(attrs, uri, opts) do
attrs
|> format_attrs()
|> format_tag(uri, opts)
end
def format_email(attrs, email, _opts) do
attrs = format_attrs(attrs)
~s(<a #{attrs}>#{email}</a>)
def format_tag(attrs, content, %{iodata: true}) do
["<a ", attrs, ">", content, "</a>"]
end
def format_extra(attrs, uri, _opts) do
attrs = format_attrs(attrs)
~s(<a #{attrs}>#{uri}</a>)
def format_tag(attrs, content, %{iodata: :safe}) do
[{:safe, ["<a ", attrs, ">"]}, content, {:safe, "</a>"}]
end
defp format_attributes(attrs) do
Enum.reduce(attrs, "", fn {name, value}, acc ->
acc <> ~s' #{name}="#{value}"'
end)
def format_tag(attrs, content, _opts) do
"<a #{attrs}>#{content}</a>"
end
end
lib/linkify/parser.ex 0 → 100644
View file @ 3f14b5b3
# Copyright © 2017-2018 E-MetroTel
# Copyright © 2019-2022 Pleroma Authors
# SPDX-License-Identifier: MIT
defmodule Linkify.Parser do
@moduledoc """
Module to handle parsing the the input string.
"""
alias Linkify.Builder
@invalid_url ~r/(\.\.+)|(^(\d+\.){1,2}\d+$)/
@match_url ~r{^(?:\W*)?(?<url>(?:https?:\/\/)?[\w.-]+(?:\.[\w\.-]+)+[\w\-\._~%:\/?#[\]@!\$&'\(\)\*\+,;=.]+$)}u
@get_scheme_host ~r{^\W*(?<scheme>https?:\/\/)?(?:[^@\n]+\\w@)?(?<host>[^:#~\/\n?]+)}u
@match_hashtag ~r/^(?<tag>\#[[:word:]_]*[[:alpha:]_·\x{200c}][[:word:]_·\p{M}\x{200c}]*)/u
@match_skipped_tag ~r/^(?<tag>(a|code|pre)).*>*/
@delimiters ~r/[,;:>?!]*$/
@en_apostrophes [
"'",
"'s",
"'ll",
"'d"
]
@prefix_extra [
"magnet:?",
"dweb://",
"dat://",
"gopher://",
"ipfs://",
"ipns://",
"irc://",
"ircs://",
"irc6://",
"mumble://",
"ssb://"
]
@tlds "./priv/tlds.txt"
|> File.read!()
|> String.split("\n", trim: true)
|> Enum.concat(["onion"])
|> MapSet.new()
@default_opts %{
url: true,
validate_tld: true
}
@doc """
Parse the given string, identifying items to link.
Parses the string, replacing the matching urls with an html link.
## Examples
iex> Linkify.Parser.parse("Check out google.com")
~s{Check out <a href="http://google.com">google.com</a>}
"""
@types [:url, :hashtag, :extra, :mention, :email]
def parse(input, opts \\ %{})
def parse(input, opts) when is_binary(input), do: {input, %{}} |> parse(opts) |> elem(0)
def parse(input, list) when is_list(list), do: parse(input, Enum.into(list, %{}))
def parse(input, opts) do
opts = Map.merge(@default_opts, opts)
{buffer, user_acc} = do_parse(input, opts, {"", [], :parsing})
if opts[:iodata] do
{buffer, user_acc}
else
{IO.iodata_to_binary(buffer), user_acc}
end
end
defp accumulate(acc, buffer),
do: [buffer | acc]
defp accumulate(acc, buffer, trailing),
do: [trailing, buffer | acc]
defp do_parse({"", user_acc}, _opts, {"", acc, _}),
do: {Enum.reverse(acc), user_acc}
defp do_parse(
{"<" <> text, user_acc},
%{hashtag: true} = opts,
{"#" <> _ = buffer, acc, :parsing}
) do
{buffer, user_acc} = link(buffer, opts, user_acc)
buffer =
case buffer do
[_, _, _] -> Enum.join(buffer)
_ -> buffer
end
case Regex.run(@match_skipped_tag, buffer, capture: [:tag]) do
[tag] ->
text = String.trim_leading(text, tag)
do_parse({text, user_acc}, opts, {"", accumulate(acc, buffer, "<#{tag}"), :skip})
nil ->
do_parse({text, user_acc}, opts, {"<", accumulate(acc, buffer, ""), {:open, 1}})
end
end
defp do_parse({"<br" <> text, user_acc}, opts, {buffer, acc, :parsing}) do
{buffer, user_acc} = link(buffer, opts, user_acc)
do_parse({text, user_acc}, opts, {"", accumulate(acc, buffer, "<br"), {:open, 1}})
end
defp do_parse({"<a" <> text, user_acc}, opts, {buffer, acc, :parsing}),
do: do_parse({text, user_acc}, opts, {"", accumulate(acc, buffer, "<a"), :skip})
defp do_parse({"<pre" <> text, user_acc}, opts, {buffer, acc, :parsing}),
do: do_parse({text, user_acc}, opts, {"", accumulate(acc, buffer, "<pre"), :skip})
defp do_parse({"<code" <> text, user_acc}, opts, {buffer, acc, :parsing}),
do: do_parse({text, user_acc}, opts, {"", accumulate(acc, buffer, "<code"), :skip})
defp do_parse({"</a>" <> text, user_acc}, opts, {buffer, acc, :skip}),
do: do_parse({text, user_acc}, opts, {"", accumulate(acc, buffer, "</a>"), :parsing})
defp do_parse({"</pre>" <> text, user_acc}, opts, {buffer, acc, :skip}),
do: do_parse({text, user_acc}, opts, {"", accumulate(acc, buffer, "</pre>"), :parsing})
defp do_parse({"</code>" <> text, user_acc}, opts, {buffer, acc, :skip}),
do: do_parse({text, user_acc}, opts, {"", accumulate(acc, buffer, "</code>"), :parsing})
defp do_parse({"<" <> text, user_acc}, opts, {"", acc, :parsing}),
do: do_parse({text, user_acc}, opts, {"<", acc, {:open, 1}})
defp do_parse({"<" <> text, user_acc}, opts, {buffer, acc, :parsing}) do
{buffer, user_acc} = link(buffer, opts, user_acc)
do_parse({text, user_acc}, opts, {"", accumulate(acc, buffer, "<"), {:open, 1}})
end
defp do_parse({">" <> text, user_acc}, opts, {buffer, acc, {:attrs, _level}}),
do: do_parse({text, user_acc}, opts, {"", accumulate(acc, buffer, ">"), :parsing})
defp do_parse({<<ch::8>> <> text, user_acc}, opts, {"", acc, {:attrs, level}}) do
do_parse({text, user_acc}, opts, {"", accumulate(acc, <<ch::8>>), {:attrs, level}})
end
defp do_parse({text, user_acc}, opts, {buffer, acc, {:open, level}}) do
do_parse({text, user_acc}, opts, {"", accumulate(acc, buffer), {:attrs, level}})
end
defp do_parse(
{<<char::bytes-size(1), text::binary>>, user_acc},
opts,
{buffer, acc, state}
)
when char in [" ", "\r", "\n"] do
{buffer, user_acc} = link(buffer, opts, user_acc)
do_parse(
{text, user_acc},
opts,
{"", accumulate(acc, buffer, char), state}
)
end
defp do_parse({<<ch::8>>, user_acc}, opts, {buffer, acc, state}) do
{buffer, user_acc} = link(buffer <> <<ch::8>>, opts, user_acc)
do_parse(
{"", user_acc},
opts,
{"", accumulate(acc, buffer), state}
)
end
defp do_parse({<<ch::8>> <> text, user_acc}, opts, {buffer, acc, state}),
do: do_parse({text, user_acc}, opts, {buffer <> <<ch::8>>, acc, state})
def check_and_link(:url, buffer, opts, _user_acc) do
if url?(buffer, opts) do
case @match_url |> Regex.run(buffer, capture: [:url]) |> hd() do
^buffer ->
link_url(buffer, opts)
url ->
link = link_url(url, opts)
restore_stripped_symbols(buffer, url, link)
end
else
:nomatch
end
end
def check_and_link(:email, buffer, opts, _user_acc) do
if email?(buffer, opts), do: link_email(buffer, opts), else: :nomatch
end
def check_and_link(:mention, buffer, opts, user_acc) do
buffer
|> match_mention
|> link_mention(buffer, opts, user_acc)
end
def check_and_link(:hashtag, buffer, opts, user_acc) do
buffer
|> match_hashtag
|> link_hashtag(buffer, opts, user_acc)
end
def check_and_link(:extra, "xmpp:" <> handle = buffer, opts, _user_acc) do
if email?(handle, opts), do: link_extra(buffer, opts), else: :nomatch
end
def check_and_link(:extra, buffer, opts, _user_acc) do
if String.starts_with?(buffer, @prefix_extra), do: link_extra(buffer, opts), else: :nomatch
end
defp maybe_strip_trailing_period(buffer, type) when type in [:mention, :hashtag, :email],
do: String.trim_trailing(buffer, ".")
defp maybe_strip_trailing_period(buffer, _), do: buffer
defp maybe_strip_parens(buffer) do
trimmed = trim_leading_paren(buffer)
with :next <- parens_check_trailing(buffer),
:next <- parens_found_email(trimmed),
:next <- parens_found_url(trimmed),
%{path: path, query: query} = URI.parse(trimmed),
:next <- parens_in_query(query),
:next <- parens_found_path_separator(path),
:next <- parens_path_has_open_paren(path),
:next <- parens_check_balanced(trimmed) do
buffer |> trim_leading_paren |> trim_trailing_paren
else
:both -> buffer |> trim_leading_paren |> trim_trailing_paren
:leading_only -> buffer |> trim_leading_paren
:noop -> buffer
_ -> buffer
end
end
defp parens_check_trailing(buffer), do: (String.ends_with?(buffer, ")") && :next) || :noop
defp parens_found_email(trimmed),
do: (trim_trailing_paren(trimmed) |> email?(nil) && :both) || :next
defp parens_found_url(trimmed),
do: (trim_trailing_paren(trimmed) |> url?(nil) && :next) || :noop
defp parens_in_query(query), do: (is_nil(query) && :next) || :both
defp parens_found_path_separator(path) when is_nil(path), do: :next
defp parens_found_path_separator(path), do: (String.contains?(path, "/") && :next) || :both
defp parens_path_has_open_paren(path) when is_nil(path), do: :next
defp parens_path_has_open_paren(path), do: (String.contains?(path, "(") && :next) || :both
defp parens_check_balanced(trimmed) do
graphemes = String.graphemes(trimmed)
opencnt = graphemes |> Enum.count(fn x -> x == "(" end)
closecnt = graphemes |> Enum.count(fn x -> x == ")" end)
if opencnt == closecnt do
:leading_only
else
:next
end
end
defp trim_leading_paren(buffer) do
case buffer do
"(" <> buffer -> buffer
buffer -> buffer
end
end
defp trim_trailing_paren(buffer),
do:
(String.ends_with?(buffer, ")") && String.slice(buffer, 0, String.length(buffer) - 1)) ||
buffer
defp strip_punctuation(buffer), do: String.replace(buffer, @delimiters, "")
defp strip_en_apostrophes(buffer) do
Enum.reduce(@en_apostrophes, buffer, fn abbrev, buf ->
String.replace_suffix(buf, abbrev, "")
end)
end
def url?(buffer, opts) do
valid_url?(buffer) && Regex.match?(@match_url, buffer) && valid_tld?(buffer, opts)
end
def email?(buffer, opts) do
# Note: In reality the local part can only be checked by the remote server
case Regex.run(~r/^(?<user>.*)@(?<host>[^@]+)$/, buffer, capture: [:user, :host]) do
[_user, hostname] -> valid_hostname?(hostname) && valid_tld?(hostname, opts)
_ -> false
end
end
defp valid_url?(url) do
with {_, [scheme]} <- {:regex, Regex.run(@get_scheme_host, url, capture: [:scheme])},
true <- scheme == "" do
!Regex.match?(@invalid_url, url)
else
_ ->
true
end
end
@doc """
Validates a URL's TLD. Returns a boolean.
Will return `true` if `:validate_tld` option set to `false`.
Will skip validation and return `true` if `:validate_tld` set to `:no_scheme` and the url has a scheme.
"""
def valid_tld?(url, opts) do
[scheme, host] = Regex.run(@get_scheme_host, url, capture: [:scheme, :host])
cond do
opts[:validate_tld] == false ->
true
scheme != "" && ip?(host) ->
true
# don't validate if scheme is present
opts[:validate_tld] == :no_scheme and scheme != "" ->
true
true ->
tld = host |> String.trim_trailing(".") |> String.split(".") |> List.last()
MapSet.member?(@tlds, tld)
end
end
def safe_to_integer(string, base \\ 10) do
String.to_integer(string, base)
rescue
_ ->
nil
end
def ip?(buffer) do
case :inet.parse_strict_address(to_charlist(buffer)) do
{:error, _} -> false
{:ok, _} -> true
end
end
# IDN-compatible, ported from musl-libc's is_valid_hostname()
def valid_hostname?(hostname) do
hostname
|> String.to_charlist()
|> Enum.any?(fn s ->
!(s >= 0x80 || s in 0x30..0x39 || s in 0x41..0x5A || s in 0x61..0x7A || s in '.-')
end)
|> Kernel.!()
end
def match_mention(buffer) do
case Regex.run(~r/^@(?<user>[a-zA-Z\d_-]+)(@(?<host>[^@]+))?$/, buffer,
capture: [:user, :host]
) do
[user, ""] ->
"@" <> user
[user, hostname] ->
if valid_hostname?(hostname) && valid_tld?(hostname, []),
do: "@" <> user <> "@" <> hostname,
else: nil
_ ->
nil
end
end
def match_hashtag(buffer) do
case Regex.run(@match_hashtag, buffer, capture: [:tag]) do
[hashtag] -> hashtag
_ -> nil
end
end
def link_hashtag(nil, _buffer, _, _user_acc), do: :nomatch
def link_hashtag(hashtag, buffer, %{hashtag_handler: hashtag_handler} = opts, user_acc) do
hashtag
|> hashtag_handler.(buffer, opts, user_acc)
|> maybe_update_buffer(hashtag, buffer)
end
def link_hashtag(hashtag, buffer, opts, _user_acc) do
hashtag
|> Builder.create_hashtag_link(buffer, opts)
|> maybe_update_buffer(hashtag, buffer)
end
def link_mention(nil, _buffer, _, _user_acc), do: :nomatch
def link_mention(mention, buffer, %{mention_handler: mention_handler} = opts, user_acc) do
mention
|> mention_handler.(buffer, opts, user_acc)
|> maybe_update_buffer(mention, buffer)
end
def link_mention(mention, buffer, opts, _user_acc) do
mention
|> Builder.create_mention_link(buffer, opts)
|> maybe_update_buffer(mention, buffer)
end
defp maybe_update_buffer(out, match, buffer) when is_binary(out) do
maybe_update_buffer({out, nil}, match, buffer)
end
defp maybe_update_buffer({out, user_acc}, match, buffer)
when match != buffer and out != buffer do
out = String.replace(buffer, match, out)
{out, user_acc}
end
defp maybe_update_buffer(out, _match, _buffer), do: out
@doc false
def link_url(buffer, opts) do
Builder.create_link(buffer, opts)
end
@doc false
def link_email(buffer, opts) do
Builder.create_email_link(buffer, opts)
end
def link_extra(buffer, opts) do
Builder.create_extra_link(buffer, opts)
end
defp link(buffer, opts, user_acc) do
Enum.reduce_while(@types, {buffer, user_acc}, fn type, _ ->
if opts[type] == true do
check_and_link_reducer(type, buffer, opts, user_acc)
else
{:cont, {buffer, user_acc}}
end
end)
end
defp check_and_link_reducer(type, buffer, opts, user_acc) do
str =
buffer
|> String.split("<")
|> List.first()
|> strip_en_apostrophes()
|> strip_punctuation()
|> maybe_strip_trailing_period(type)
|> maybe_strip_parens()
case check_and_link(type, str, opts, user_acc) do
:nomatch ->
{:cont, {buffer, user_acc}}
{link, user_acc} ->
{:halt, {restore_stripped_symbols(buffer, str, link), user_acc}}
link ->
{:halt, {restore_stripped_symbols(buffer, str, link), user_acc}}
end
end
defp restore_stripped_symbols(buffer, buffer, link), do: link
defp restore_stripped_symbols(buffer, stripped_buffer, link) do
buffer
|> String.split(stripped_buffer)
|> Enum.intersperse(link)
end
end
mix.exs
View file @ 3f14b5b3
defmodule AutoLinker.Mixfile do
# Copyright © 2017-2018 E-MetroTel
# Copyright © 2019-2022 Pleroma Authors
# SPDX-License-Identifier: MIT
defmodule Linkify.Mixfile do
use Mix.Project
@version "0.2.2"
@version "0.5.3"
def project do
[
app: :auto_linker,
app: :linkify,
version: @version,
elixir: "~> 1.4",
elixir: "~> 1.8",
build_embedded: Mix.env() == :prod,
start_permanent: Mix.env() == :prod,
aliases: aliases(),
deps: deps(),
docs: [extras: ["README.md"]],
package: package(),
name: "AutoLinker",
name: "Linkify",
description: """
AutoLinker is a basic package for turning website names into links.
Linkify is a basic package for turning website names into links.
"""
]
end
......@@ -29,18 +34,30 @@ defmodule AutoLinker.Mixfile do
# Dependencies can be Hex packages:
defp deps do
[
{:ex_doc, "~> 0.19", only: :dev, runtime: false},
{:earmark, "~> 1.2", only: :dev, override: true},
{:credo, "~> 1.0.0", only: [:dev, :test], runtime: false}
{:ex_doc, "~> 0.20", only: :dev, runtime: false},
{:credo, "~> 1.5", only: [:dev, :test], runtime: false}
]
end
defp package do
[
maintainers: ["Stephen Pallen"],
licenses: ["MIT"],
links: %{"Github" => "https://github.com/smpallen99/auto_linker"},
files: ~w(lib README.md mix.exs LICENSE)
licenses: ["MIT", "CC0-1.0"],
links: %{"GitLab" => "https://git.pleroma.social/pleroma/elixir-libraries/linkify"},
files: ~w(lib priv README.md mix.exs)
]
end
defp aliases do
[
"update.tlds": &update_tlds/1
]
end
defp update_tlds(_) do
:os.cmd(
String.to_charlist(
"curl https://data.iana.org/TLD/tlds-alpha-by-domain.txt | tr '[:upper:]' '[:lower:]' | tail -n +2 > priv/tlds.txt"
)
)
end
end
mix.lock
View file @ 3f14b5b3
%{
"bunt": {:hex, :bunt, "0.2.0", "951c6e801e8b1d2cbe58ebbd3e616a869061ddadcc4863d0a2182541acae9a38", [:mix], [], "hexpm"},
"credo": {:hex, :credo, "1.0.2", "88bc918f215168bf6ce7070610a6173c45c82f32baa08bdfc80bf58df2d103b6", [:mix], [{:bunt, "~> 0.2.0", [hex: :bunt, repo: "hexpm", optional: false]}, {:jason, "~> 1.0", [hex: :jason, repo: "hexpm", optional: false]}], "hexpm"},
"earmark": {:hex, :earmark, "1.3.1", "73812f447f7a42358d3ba79283cfa3075a7580a3a2ed457616d6517ac3738cb9", [:mix], [], "hexpm"},
"ex_doc": {:hex, :ex_doc, "0.19.3", "3c7b0f02851f5fc13b040e8e925051452e41248f685e40250d7e40b07b9f8c10", [:mix], [{:earmark, "~> 1.2", [hex: :earmark, repo: "hexpm", optional: false]}, {:makeup_elixir, "~> 0.10", [hex: :makeup_elixir, repo: "hexpm", optional: false]}], "hexpm"},
"file_system": {:hex, :file_system, "0.2.6", "fd4dc3af89b9ab1dc8ccbcc214a0e60c41f34be251d9307920748a14bf41f1d3", [:mix], [], "hexpm"},
"jason": {:hex, :jason, "1.1.2", "b03dedea67a99223a2eaf9f1264ce37154564de899fd3d8b9a21b1a6fd64afe7", [:mix], [{:decimal, "~> 1.0", [hex: :decimal, repo: "hexpm", optional: true]}], "hexpm"},
"makeup": {:hex, :makeup, "0.8.0", "9cf32aea71c7fe0a4b2e9246c2c4978f9070257e5c9ce6d4a28ec450a839b55f", [:mix], [{:nimble_parsec, "~> 0.5.0", [hex: :nimble_parsec, repo: "hexpm", optional: false]}], "hexpm"},
"makeup_elixir": {:hex, :makeup_elixir, "0.13.0", "be7a477997dcac2e48a9d695ec730b2d22418292675c75aa2d34ba0909dcdeda", [:mix], [{:makeup, "~> 0.8", [hex: :makeup, repo: "hexpm", optional: false]}], "hexpm"},
"mix_test_watch": {:hex, :mix_test_watch, "0.9.0", "c72132a6071261893518fa08e121e911c9358713f62794a90c95db59042af375", [:mix], [{:file_system, "~> 0.2.1 or ~> 0.3", [hex: :file_system, repo: "hexpm", optional: false]}], "hexpm"},
"nimble_parsec": {:hex, :nimble_parsec, "0.5.0", "90e2eca3d0266e5c53f8fbe0079694740b9c91b6747f2b7e3c5d21966bba8300", [:mix], [], "hexpm"},
"bunt": {:hex, :bunt, "0.2.0", "951c6e801e8b1d2cbe58ebbd3e616a869061ddadcc4863d0a2182541acae9a38", [:mix], [], "hexpm", "7af5c7e09fe1d40f76c8e4f9dd2be7cebd83909f31fee7cd0e9eadc567da8353"},
"credo": {:hex, :credo, "1.6.4", "ddd474afb6e8c240313f3a7b0d025cc3213f0d171879429bf8535d7021d9ad78", [:mix], [{:bunt, "~> 0.2.0", [hex: :bunt, repo: "hexpm", optional: false]}, {:file_system, "~> 0.2.8", [hex: :file_system, repo: "hexpm", optional: false]}, {:jason, "~> 1.0", [hex: :jason, repo: "hexpm", optional: false]}], "hexpm", "c28f910b61e1ff829bffa056ef7293a8db50e87f2c57a9b5c3f57eee124536b7"},
"earmark": {:hex, :earmark, "1.4.10", "bddce5e8ea37712a5bfb01541be8ba57d3b171d3fa4f80a0be9bcf1db417bcaf", [:mix], [{:earmark_parser, ">= 1.4.10", [hex: :earmark_parser, repo: "hexpm", optional: false]}], "hexpm", "12dbfa80810478e521d3ffb941ad9fbfcbbd7debe94e1341b4c4a1b2411c1c27"},
"earmark_parser": {:hex, :earmark_parser, "1.4.26", "f4291134583f373c7d8755566122908eb9662df4c4b63caa66a0eabe06569b0a", [:mix], [], "hexpm", "48d460899f8a0c52c5470676611c01f64f3337bad0b26ddab43648428d94aabc"},
"ex_doc": {:hex, :ex_doc, "0.28.4", "001a0ea6beac2f810f1abc3dbf4b123e9593eaa5f00dd13ded024eae7c523298", [:mix], [{:earmark_parser, "~> 1.4.19", [hex: :earmark_parser, repo: "hexpm", optional: false]}, {:makeup_elixir, "~> 0.14", [hex: :makeup_elixir, repo: "hexpm", optional: false]}, {:makeup_erlang, "~> 0.1", [hex: :makeup_erlang, repo: "hexpm", optional: false]}], "hexpm", "bf85d003dd34911d89c8ddb8bda1a958af3471a274a4c2150a9c01c78ac3f8ed"},
"file_system": {:hex, :file_system, "0.2.10", "fb082005a9cd1711c05b5248710f8826b02d7d1784e7c3451f9c1231d4fc162d", [:mix], [], "hexpm", "41195edbfb562a593726eda3b3e8b103a309b733ad25f3d642ba49696bf715dc"},
"jason": {:hex, :jason, "1.3.0", "fa6b82a934feb176263ad2df0dbd91bf633d4a46ebfdffea0c8ae82953714946", [:mix], [{:decimal, "~> 1.0 or ~> 2.0", [hex: :decimal, repo: "hexpm", optional: true]}], "hexpm", "53fc1f51255390e0ec7e50f9cb41e751c260d065dcba2bf0d08dc51a4002c2ac"},
"makeup": {:hex, :makeup, "1.1.0", "6b67c8bc2882a6b6a445859952a602afc1a41c2e08379ca057c0f525366fc3ca", [:mix], [{:nimble_parsec, "~> 1.2.2 or ~> 1.3", [hex: :nimble_parsec, repo: "hexpm", optional: false]}], "hexpm", "0a45ed501f4a8897f580eabf99a2e5234ea3e75a4373c8a52824f6e873be57a6"},
"makeup_elixir": {:hex, :makeup_elixir, "0.16.0", "f8c570a0d33f8039513fbccaf7108c5d750f47d8defd44088371191b76492b0b", [:mix], [{:makeup, "~> 1.0", [hex: :makeup, repo: "hexpm", optional: false]}, {:nimble_parsec, "~> 1.2.3", [hex: :nimble_parsec, repo: "hexpm", optional: false]}], "hexpm", "28b2cbdc13960a46ae9a8858c4bebdec3c9a6d7b4b9e7f4ed1502f8159f338e7"},
"makeup_erlang": {:hex, :makeup_erlang, "0.1.1", "3fcb7f09eb9d98dc4d208f49cc955a34218fc41ff6b84df7c75b3e6e533cc65f", [:mix], [{:makeup, "~> 1.0", [hex: :makeup, repo: "hexpm", optional: false]}], "hexpm", "174d0809e98a4ef0b3309256cbf97101c6ec01c4ab0b23e926a9e17df2077cbb"},
"nimble_parsec": {:hex, :nimble_parsec, "1.2.3", "244836e6e3f1200c7f30cb56733fd808744eca61fd182f731eac4af635cc6d0b", [:mix], [], "hexpm", "c8d789e39b9131acf7b99291e93dae60ab48ef14a7ee9d58c6964f59efb570b0"},
}
mix.lock.license 0 → 100644
View file @ 3f14b5b3
# Copyright © 2017-2018 E-MetroTel
# Copyright © 2019-2022 Pleroma Authors
# SPDX-License-Identifier: MIT
priv/tlds.txt
View file @ 3f14b5b3
......@@ -14,10 +14,8 @@ accenture
accountant
accountants
aco
active
actor
ad
adac
ads
adult
ae
......@@ -25,7 +23,6 @@ aeg
aero
aetna
af
afamilycompany
afl
africa
ag
......@@ -33,7 +30,6 @@ agakhan
agency
ai
aig
aigo
airbus
airforce
airtel
......@@ -48,6 +44,7 @@ ally
alsace
alstom
am
amazon
americanexpress
americanfamily
amex
......@@ -144,7 +141,6 @@ biz
bj
black
blackfriday
blanco
blockbuster
blog
bloomberg
......@@ -153,7 +149,6 @@ bm
bms
bmw
bn
bnl
bnpparibas
bo
boats
......@@ -164,7 +159,6 @@ bond
boo
book
booking
boots
bosch
bostik
boston
......@@ -180,8 +174,6 @@ brother
brussels
bs
bt
budapest
bugatti
build
builders
business
......@@ -201,7 +193,6 @@ calvinklein
cam
camera
camp
cancerresearch
canon
capetown
capital
......@@ -213,10 +204,8 @@ care
career
careers
cars
cartier
casa
case
caseih
cash
casino
cat
......@@ -228,7 +217,6 @@ cbre
cbs
cc
cd
ceb
center
ceo
cern
......@@ -239,14 +227,13 @@ cg
ch
chanel
channel
charity
chase
chat
cheap
chintai
chloe
christmas
chrome
chrysler
church
ci
cipriani
......@@ -298,6 +285,7 @@ country
coupon
coupons
courses
cpa
cr
credit
creditcard
......@@ -307,7 +295,6 @@ crown
crs
cruise
cruises
csc
cu
cuisinella
cv
......@@ -359,18 +346,14 @@ dnp
do
docs
doctor
dodge
dog
doha
domains
dot
download
drive
dtv
dubai
duck
dunlop
duns
dupont
durban
dvag
......@@ -391,7 +374,6 @@ energy
engineer
engineering
enterprises
epost
epson
equipment
er
......@@ -400,14 +382,12 @@ erni
es
esq
estate
esurance
et
etisalat
eu
eurovision
eus
events
everbank
exchange
expert
exposed
......@@ -472,7 +452,6 @@ frontdoor
frontier
ftr
fujitsu
fujixerox
fun
fund
furniture
......@@ -487,6 +466,7 @@ game
games
gap
garden
gay
gb
gbiz
gd
......@@ -506,7 +486,6 @@ gifts
gives
giving
gl
glade
glass
gle
global
......@@ -522,7 +501,6 @@ gold
goldpoint
golf
goo
goodhands
goodyear
goog
google
......@@ -580,7 +558,6 @@ homegoods
homes
homesense
honda
honeywell
horse
hospital
host
......@@ -594,7 +571,6 @@ how
hr
hsbc
ht
htc
hu
hughes
hyatt
......@@ -615,6 +591,7 @@ imdb
immo
immobilien
in
inc
industries
infiniti
info
......@@ -624,7 +601,6 @@ institute
insurance
insure
int
intel
international
intuit
investments
......@@ -634,25 +610,20 @@ iq
ir
irish
is
iselect
ismaili
ist
istanbul
it
itau
itv
iveco
iwc
jaguar
java
jcb
jcp
je
jeep
jetzt
jewelry
jio
jlc
jll
jm
jmp
......@@ -678,6 +649,7 @@ kg
kh
ki
kia
kids
kim
kinder
kindle
......@@ -701,12 +673,10 @@ kyoto
kz
la
lacaixa
ladbrokes
lamborghini
lamer
lancaster
lancia
lancome
land
landrover
lanxess
......@@ -727,7 +697,6 @@ lego
lexus
lgbt
li
liaison
lidl
life
lifeinsurance
......@@ -743,13 +712,13 @@ link
lipsy
live
living
lixil
lk
llc
llp
loan
loans
locker
locus
loft
lol
london
lotte
......@@ -764,13 +733,11 @@ ltd
ltda
lu
lundbeck
lupin
luxe
luxury
lv
ly
ma
macys
madrid
maif
maison
......@@ -799,9 +766,7 @@ meme
memorial
men
menu
meo
merckmsd
metlife
mg
mh
miami
......@@ -821,7 +786,6 @@ mn
mo
mobi
mobile
mobily
moda
moe
moi
......@@ -829,7 +793,6 @@ mom
monash
money
monster
mopar
mormon
mortgage
moscow
......@@ -837,7 +800,6 @@ moto
motorcycles
mov
movie
movistar
mp
mq
mr
......@@ -848,6 +810,7 @@ mtn
mtr
mu
museum
music
mutual
mv
mw
......@@ -856,10 +819,8 @@ my
mz
na
nab
nadex
nagoya
name
nationwide
natura
navy
nba
......@@ -872,7 +833,6 @@ netflix
network
neustar
new
newholland
news
next
nextdirect
......@@ -907,7 +867,6 @@ nyc
nz
obi
observer
off
office
okinawa
olayan
......@@ -920,7 +879,6 @@ one
ong
onl
online
onyourside
ooo
open
oracle
......@@ -935,7 +893,6 @@ ovh
pa
page
panasonic
panerai
paris
pars
partners
......@@ -958,7 +915,6 @@ photo
photography
photos
physio
piaget
pics
pictet
pictures
......@@ -1009,10 +965,8 @@ qa
qpon
quebec
quest
qvc
racing
radio
raid
re
read
realestate
......@@ -1041,11 +995,9 @@ rexroth
rich
richardli
ricoh
rightathome
ril
rio
rip
rmit
ro
rocher
rocks
......@@ -1074,7 +1026,6 @@ sandvik
sandvikcoromant
sanofi
sap
sapo
sarl
sas
save
......@@ -1092,8 +1043,6 @@ school
schule
schwarz
science
scjohnson
scor
scot
sd
se
......@@ -1105,7 +1054,6 @@ seek
select
sener
services
ses
seven
sew
sex
......@@ -1125,7 +1073,6 @@ shopping
shouji
show
showtime
shriram
si
silk
sina
......@@ -1155,21 +1102,19 @@ solutions
song
sony
soy
spa
space
spiegel
sport
spot
spreadbetting
sr
srl
srt
ss
st
stada
staples
star
starhub
statebank
statefarm
statoil
stc
stcgroup
stockholm
......@@ -1189,12 +1134,10 @@ surgery
suzuki
sv
swatch
swiftcover
swiss
sx
sy
sydney
symantec
systems
sz
tab
......@@ -1215,8 +1158,6 @@ team
tech
technology
tel
telecity
telefonica
temasek
tennis
teva
......@@ -1276,7 +1217,6 @@ tz
ua
ubank
ubs
uconnect
ug
uk
unicom
......@@ -1310,8 +1250,6 @@ vip
virgin
visa
vision
vista
vistaprint
viva
vivo
vlaanderen
......@@ -1330,7 +1268,6 @@ walmart
walter
wang
wanggou
warman
watch
watches
weather
......@@ -1375,12 +1312,12 @@ xn--3bst00m
xn--3ds443g
xn--3e0b707e
xn--3hcrj9c
xn--3oq18vl8pn36a
xn--3pxu8k
xn--42c2d9a
xn--45br5cyl
xn--45brj9c
xn--45q11c
xn--4dbrk0ce
xn--4gbrim
xn--54b7fta0cc
xn--55qw42g
......@@ -1406,6 +1343,7 @@ xn--bck1b9a5dre4c
xn--c1avg
xn--c2br7g
xn--cck2b3b
xn--cckwcxetd
xn--cg4bki
xn--clchc0ea0b2g2a9gcd
xn--czr694b
......@@ -1416,7 +1354,6 @@ xn--d1alf
xn--e1a4c
xn--eckvdtc9d
xn--efvy88h
xn--estv75g
xn--fct429k
xn--fhbei
xn--fiq228c5hs
......@@ -1442,12 +1379,11 @@ xn--io0a7i
xn--j1aef
xn--j1amh
xn--j6w193g
xn--jlq61u9w7b
xn--jlq480n2rg
xn--jvr189m
xn--kcrx77d1x4a
xn--kprw13d
xn--kpry57d
xn--kpu716f
xn--kput3i
xn--l1acc
xn--lgbbat1ad8j
......@@ -1458,13 +1394,14 @@ xn--mgba7c0bbn0a
xn--mgbaakc7dvf
xn--mgbaam7a8h
xn--mgbab2bd
xn--mgbah1a3hjkrd
xn--mgbai9azgqp6j
xn--mgbayh7gpa
xn--mgbb9fbpob
xn--mgbbh1a
xn--mgbbh1a71e
xn--mgbc0a9azcg
xn--mgbca7dzdo
xn--mgbcpq6gpa1a
xn--mgberp4a5d4ar
xn--mgbgu82a
xn--mgbi4ecexp
......@@ -1484,13 +1421,15 @@ xn--nqv7fs00ema
xn--nyqy26a
xn--o3cw4h
xn--ogbpf8fl
xn--otu796d
xn--p1acf
xn--p1ai
xn--pbt977c
xn--pgbs0dh
xn--pssy2u
xn--q7ce6a
xn--q9jyb4c
xn--qcka1pmc
xn--qxa6a
xn--qxam
xn--rhqv96g
xn--rovu88b
......@@ -1516,7 +1455,6 @@ xn--y9a3aq
xn--yfro4i67o
xn--ygbi2ammx
xn--zfr164b
xperia
xxx
xyz
yachts
......@@ -1536,7 +1474,6 @@ zappos
zara
zero
zip
zippo
zm
zone
zuerich
......
priv/tlds.txt.license 0 → 100644
View file @ 3f14b5b3
# Copyright © 2019-2020 IANA.org
# SPDX-License-Identifier: CC0-1.0
# https://www.iana.org/help/licensing-terms
# https://data.iana.org/TLD/tlds-alpha-by-domain.txt