Merge branch 'feature/add-pelias-geocoder' into 'master'

Feature/add pelias geocoder

See merge request framasoft/mobilizon!324
This commit is contained in:
Thomas Citharel 2019-11-19 20:19:02 +01:00
commit 83fcf2b62e
14 changed files with 226 additions and 27 deletions

View file

@ -140,6 +140,9 @@ config :mobilizon, Mobilizon.Service.Geospatial.MapQuest,
config :mobilizon, Mobilizon.Service.Geospatial.Mimirsbrunn,
endpoint: System.get_env("GEOSPATIAL_MIMIRSBRUNN_ENDPOINT") || nil
config :mobilizon, Mobilizon.Service.Geospatial.Pelias,
endpoint: System.get_env("GEOSPATIAL_PELIAS_ENDPOINT") || nil
config :mobilizon, Oban,
repo: Mobilizon.Storage.Repo,
prune: {:maxlen, 10_000},

View file

@ -0,0 +1,78 @@
# Geocoders
Geocoding is the ability to match an input **string representing a location - such as an address - to geographical coordinates**.
Reverse geocoding is logically the opposite, matching **geographical coordinates to names of places**.
This is needed to set correct address for events, and more easily find events with geographical data, for instance if you want to discover events happening near your current position.
However, providing a geocoding service is quite expensive, especially if you want to cover the whole Earth.
!!!note
To give an idea of what hardware is required to self-host a geocoding service, we successfully used Addok, Pelias and Mimirsbrunn on a 8 core/16GB RAM machine without any issues **on French data**.
## List of supported geocoders
This is the list of all geocoders supported by Mobilizon. The current default one is [Nominatim](#nominatim) and uses the official OpenStreetMap instance.
!!! bug
Changing geocoder through `.env` configuration isn't currently supported by Mobilizon.
Instead you need to edit the following line in `config.prod.exs`:
```elixir
config :mobilizon, Mobilizon.Service.Geospatial, service: Mobilizon.Service.Geospatial.Nominatim
```
And change `Nominatim` to one of the supported geocoders. This change might be overwritten when updating Mobilizon.
### Nominatim
[Nominatim](https://wiki.openstreetmap.org/wiki/Nominatim) is a GPL-2.0 licenced tool to search data by name and address. It's written in C and PHP and uses PostgreSQL.
It's the current default search tool on the [OpenStreetMap homepage](https://www.openstreetmap.org).
!!! warning
When using the official Nominatim OpenStreetMap instance (default endpoint for this geocoder if not configured otherwise), you need to read and accept the [Usage Policy](https://operations.osmfoundation.org/policies/nominatim).
Several companies provide hosted instances of Nominatim that you can query via an API, for example see [MapQuest Open Initiative](https://developer.mapquest.com/documentation/open/nominatim-search).
### Addok
[Addok](https://github.com/addok/addok) is a WTFPL licenced search engine for address (and only address). It's written in Python and uses Redis.
It's used by French government for [adresse.data.gouv.fr](https://adresse.data.gouv.fr).
!!! warning
When using France's Addok instance at `api-adresse.data.gouv.fr` (default endpoint for this geocoder if not configured otherwise), you need to read and accept the [GCU](https://adresse.data.gouv.fr/cgu) (in French).
### Photon
[Photon](https://photon.komoot.de/) is an Apache 2.0 licenced search engine written in Java and powered by ElasticSearch.
!!! warning
The terms of use for the official instance (default endpoint for this geocoder if not configured otherwise) are simply the following:
> You can use the API for your project, but please be fair - extensive usage will be throttled. We do not guarantee for the availability and usage might be subject of change in the future.
### Pelias
[Pelias](https://github.com/pelias/pelias) is a MIT licensed geocoder composed of several services written in NodeJS. It's powered by ElasticSearch.
There's [Geocode Earth](https://geocode.earth/) SAAS that provides a Pelias API.
They offer discounts for Open-Source projects. [See the pricing](https://geocode.earth/).
### Mimirsbrunn
[Mimirsbrunn](https://github.com/CanalTP/mimirsbrunn) is an AGPL-3.0 licensed geocoding written in Rust and powered by ElasticSearch.
Mimirsbrunn is used by [Qwant Maps](https://www.qwant.com/maps) and [Navitia](https://www.navitia.io).
### Google Maps
[Google Maps](https://developers.google.com/maps/documentation/geocoding/intro) is a proprietary service that provides APIs for geocoding.
They don't have a free plan, but offer credit when creating a new account. [See the pricing](https://cloud.google.com/maps-platform/pricing/).
### MapQuest
[MapQuest](https://developer.mapquest.com/documentation/open/geocoding-api/) is a proprietary service that provides APIs for geocoding.
They offer a free plan. [See the pricing](https://developer.mapquest.com/plans).
### More geocoding services
Geocoding implementations are simple modules that need to implement the [`Mobilizon.Service.Geospatial.Provider` behaviour](https://framasoft.frama.io/mobilizon/backend/Mobilizon.Service.Geospatial.Provider.html), so feel free to write your own!

View file

@ -25,7 +25,8 @@
<template slot="empty">
<span v-if="isFetching">{{ $t('Searching') }}</span>
<div v-else class="is-enabled">
<span>{{ $t('No results for "{queryText}". You can try another search term or drag and drop the marker on the map', { queryText }) }}</span>
<span>{{ $t('No results for "{queryText}"') }}</span>
<span>{{ $t('You can try another search term or drag and drop the marker on the map', { queryText }) }}</span>
<!-- <p class="control" @click="openNewAddressModal">-->
<!-- <button type="button" class="button is-primary">{{ $t('Add') }}</button>-->
<!-- </p>-->

View file

@ -71,7 +71,6 @@
"Didn't receive the instructions ?": "Bestätigung nicht erhalten?",
"Display name": "Namen einzeigen",
"Display participation price": "Teilnahmegebühr anzeigen",
"Displayed name": "Angezeigter Name",
"Draft": "Entwurf",
"Drafts": "Entwürfe",
"Edit": "Bearbeiten",
@ -162,7 +161,6 @@
"No group found": "Keine Gruppe gefunden",
"No groups found": "Keine Gruppen gefunden",
"No results for \"{queryText}\"": "Keine Ergebnisse für \"{queryText}\"",
"No results for \"{queryText}\". You can try another search term or drag and drop the marker on the map": "Keine Ergebinsse für \"{queryText}\". Du kannst es erneut mit anderen Begriffen versuchen, oder mit Drag&Drop den Marker auf die Karte setzen",
"No user account with this email was found. Maybe you made a typo?": "Kein Account mit dieser E-Mail gefunden. Vielleicht hast Du dich vertippt?",
"Number of places": "Anzahl der Plätze",
"OK": "OK",

View file

@ -71,7 +71,6 @@
"Didn't receive the instructions ?": "Didn't receive the instructions ?",
"Display name": "Display name",
"Display participation price": "Display participation price",
"Displayed name": "Displayed name",
"Draft": "Draft",
"Drafts": "Drafts",
"Edit": "Edit",
@ -161,7 +160,7 @@
"No events found": "No events found",
"No group found": "No group found",
"No groups found": "No groups found",
"No results for \"{queryText}\". You can try another search term or drag and drop the marker on the map": "No results for \"{queryText}\". You can try another search term or drag and drop the marker on the map",
"No results for \"{queryText}\"": "No results for \"{queryText}\"",
"No user account with this email was found. Maybe you made a typo?": "No user account with this email was found. Maybe you made a typo?",
"Number of places": "Number of places",
"OK": "OK",
@ -172,6 +171,7 @@
"On {date}": "On {date}",
"One person is going": "No one is going | One person is going | {approved} persons are going",
"Only accessible through link and search (private)": "Only accessible through link and search (private)",
"Only alphanumeric characters and underscores are supported.": "Only alphanumeric characters and underscores are supported.",
"Opened reports": "Opened reports",
"Organized by {name}": "Organized by {name}",
"Organized": "Organized",
@ -298,6 +298,7 @@
"You are already a participant of this event.": "You are already a participant of this event.",
"You are already logged-in.": "You are already logged-in.",
"You can add tags by hitting the Enter key or by adding a comma": "You can add tags by hitting the Enter key or by adding a comma",
"You can try another search term or drag and drop the marker on the map": "You can try another search term or drag and drop the marker on the map",
"You can't remove your last identity.": "You can't remove your last identity.",
"You have been disconnected": "You have been disconnected",
"You have cancelled your participation": "You have cancelled your participation",

View file

@ -71,7 +71,6 @@
"Didn't receive the instructions ?": "Vous n'avez pas reçu les instructions ?",
"Display name": "Nom affiché",
"Display participation price": "Afficher un prix de participation",
"Displayed name": "Nom affiché",
"Draft": "Brouillon",
"Drafts": "Brouillons",
"Edit": "Éditer",
@ -162,7 +161,6 @@
"No group found": "Aucun groupe trouvé",
"No groups found": "Aucun groupe trouvé",
"No results for \"{queryText}\"": "Pas de résultats pour « {queryText} »",
"No results for \"{queryText}\". You can try another search term or drag and drop the marker on the map": "Pas de résultats pour « {queryText} ». Vous pouvez essayer avec d'autres termes de recherche ou bien glisser et déposer le marqueur sur la carte",
"No user account with this email was found. Maybe you made a typo?": "Aucun compte utilisateur trouvé pour cet email. Peut-être avez-vous fait une faute de frappe ?",
"Number of places": "Nombre de places",
"OK": "OK",
@ -173,6 +171,7 @@
"On {date}": "Le {date}",
"One person is going": "Personne n'y va | Une personne y va | {approved} personnes y vont",
"Only accessible through link and search (private)": "Uniquement accessibles par lien et la recherche (privé)",
"Only alphanumeric characters and underscores are supported.": "Seuls les caractères alphanumériques et les tirets bas sont acceptés.",
"Opened reports": "Signalements ouverts",
"Organized by {name}": "Organisé par {name}",
"Organized": "Organisés",
@ -300,6 +299,7 @@
"You are already a participant of this event.": "Vous participez déjà à cet événement.",
"You are already logged-in.": "Vous êtes déjà connecté.",
"You can add tags by hitting the Enter key or by adding a comma": "Vous pouvez ajouter des tags en appuyant sur la touche Entrée ou bien en ajoutant une virgule",
"You can try another search term or drag and drop the marker on the map": "Vous pouvez essayer avec d'autres termes de recherche ou bien glisser et déposer le marqueur sur la carte",
"You can't remove your last identity.": "Vous ne pouvez pas supprimer votre dernière identité.",
"You have been disconnected": "Vous avez été déconnecté⋅e",
"You have cancelled your participation": "Vous avez annulé votre participation",

View file

@ -71,7 +71,6 @@
"Didn't receive the instructions ?": "Hebt u de instructies niet ontvangen?",
"Display name": "Getoonde naam",
"Display participation price": "Prijs voor deelname tonen",
"Displayed name": "Getoonde naam",
"Draft": "Concept",
"Drafts": "Concepten",
"Edit": "Bewerken",

View file

@ -76,7 +76,6 @@
"Disallow promoting on Mobilizon": "Refusar la promocion sus Mobilizon",
"Display name": "Nom mostrat",
"Display participation price": "Far veire un prètz de participacion",
"Displayed name": "Nom mostrat",
"Do you want to participate in {title}?": "Volètz participar a {title} ?",
"Draft": "Borrolhon",
"Drafts": "Borrolhons",
@ -184,7 +183,6 @@
"No groups found": "Cap de grop pas trobat",
"No participants yet": "Cap de participant pel moment",
"No results for \"{queryText}\"": "Cap de resultats per « {queryText} »",
"No results for \"{queryText}\". You can try another search term or drag and drop the marker on the map": "Cap de resultat per « {queryText} ». Podètz ensajar un autre tèrme de recèrca o botar lo marcador sus la mapa",
"No user account with this email was found. Maybe you made a typo?": "Pas de compte utilizaire pas trobat amb aquesta adreça. Benlèu quavètz fach una deca ?",
"Number of places": "Nombre de plaças",
"OK": "OK",

View file

@ -69,7 +69,6 @@
"Didn't receive the instructions ?": "Nie otrzymałeś(-aś) instrukcji?",
"Display name": "Wyświetlana nazwa",
"Display participation price": "Wyświetlaj cenę udziału",
"Displayed name": "Wyświetlana nazwa",
"Draft": "Szkic",
"Drafts": "Szkice",
"Edit": "Edytuj",

View file

@ -71,7 +71,6 @@
"Didn't receive the instructions ?": "Fick inte instruktionerna?",
"Display name": "Visa namn",
"Display participation price": "Visa pris för deltagande",
"Displayed name": "Visat namn",
"Draft": "Utkast",
"Drafts": "Utkast",
"Edit": "Redigera",

View file

@ -81,8 +81,10 @@ export class Address implements IAddress {
}
} else if (this.locality && this.locality.trim()) {
alternativeName = `${this.locality}, ${this.region}, ${this.country}`;
} else {
} else if (this.region && this.region.trim()) {
alternativeName = `${this.region}, ${this.country}`;
} else if (this.country && this.country.trim()) {
alternativeName = this.country;
}
poiIcon = this.iconForPOI;
break;

View file

@ -0,0 +1,128 @@
defmodule Mobilizon.Service.Geospatial.Pelias do
@moduledoc """
[Pelias](https://pelias.io) backend.
Doesn't provide type of POI.
"""
alias Mobilizon.Addresses.Address
alias Mobilizon.Service.Geospatial.Provider
alias Mobilizon.Config
require Logger
@behaviour Provider
@endpoint Application.get_env(:mobilizon, __MODULE__) |> get_in([:endpoint])
@impl Provider
@doc """
Pelias implementation for `c:Mobilizon.Service.Geospatial.Provider.geocode/3`.
"""
@spec geocode(number(), number(), keyword()) :: list(Address.t())
def geocode(lon, lat, options \\ []) do
user_agent = Keyword.get(options, :user_agent, Config.instance_user_agent())
headers = [{"User-Agent", user_agent}]
url = build_url(:geocode, %{lon: lon, lat: lat}, options)
Logger.debug("Asking Pelias for reverse geocoding with #{url}")
with {:ok, %HTTPoison.Response{status_code: 200, body: body}} <-
HTTPoison.get(url, headers),
{:ok, %{"features" => features}} <- Poison.decode(body) do
process_data(features)
end
end
@impl Provider
@doc """
Pelias implementation for `c:Mobilizon.Service.Geospatial.Provider.search/2`.
"""
@spec search(String.t(), keyword()) :: list(Address.t())
def search(q, options \\ []) do
user_agent = Keyword.get(options, :user_agent, Config.instance_user_agent())
headers = [{"User-Agent", user_agent}]
url = build_url(:search, %{q: q}, options)
Logger.debug("Asking Pelias for addresses with #{url}")
with {:ok, %HTTPoison.Response{status_code: 200, body: body}} <-
HTTPoison.get(url, headers),
{:ok, %{"features" => features}} <- Poison.decode(body) do
process_data(features)
end
end
@spec build_url(atom(), map(), list()) :: String.t()
defp build_url(method, args, options) do
limit = Keyword.get(options, :limit, 10)
lang = Keyword.get(options, :lang, "en")
coords = Keyword.get(options, :coords, nil)
endpoint = Keyword.get(options, :endpoint, @endpoint)
country_code = Keyword.get(options, :country_code)
url =
case method do
:search ->
url =
"#{endpoint}/v1/autocomplete?text=#{URI.encode(args.q)}&lang=#{lang}&size=#{limit}"
if is_nil(coords),
do: url,
else: url <> "&focus.point.lat=#{coords.lat}&focus.point.lon=#{coords.lon}"
:geocode ->
"#{endpoint}/v1/reverse?point.lon=#{args.lon}&point.lat=#{args.lat}"
end
if is_nil(country_code), do: url, else: "#{url}&boundary.country=#{country_code}"
end
defp process_data(features) do
features
|> Enum.map(fn %{
"geometry" => %{"coordinates" => coordinates},
"properties" => properties
} ->
address = process_address(properties)
%Address{address | geom: Provider.coordinates(coordinates)}
end)
end
defp process_address(properties) do
%Address{
country: Map.get(properties, "country"),
locality: Map.get(properties, "locality"),
region: Map.get(properties, "region"),
description: Map.get(properties, "name"),
postal_code: Map.get(properties, "postalcode"),
street: street_address(properties),
origin_id: "pelias:#{Map.get(properties, "id")}",
type: get_type(properties)
}
end
defp street_address(properties) do
if Map.has_key?(properties, "housenumber") do
"#{Map.get(properties, "housenumber")} #{Map.get(properties, "street")}"
else
Map.get(properties, "street")
end
end
@administrative_layers [
"neighbourhood",
"borough",
"localadmin",
"locality",
"county",
"macrocounty",
"region",
"macroregion",
"dependency"
]
defp get_type(%{"layer" => layer}) when layer in @administrative_layers, do: "administrative"
defp get_type(%{"layer" => "address"}), do: "house"
defp get_type(%{"layer" => "street"}), do: "street"
defp get_type(%{"layer" => "venue"}), do: "venue"
defp get_type(%{"layer" => _}), do: nil
end

View file

@ -9,11 +9,13 @@ defmodule Mobilizon.Service.Geospatial.Provider do
* `Mobilizon.Service.Geospatial.Addok` [🔗](https://github.com/addok/addok)
* `Mobilizon.Service.Geospatial.MapQuest` [🔗](https://developer.mapquest.com/documentation/open/)
* `Mobilizon.Service.Geospatial.GoogleMaps` [🔗](https://developers.google.com/maps/documentation/geocoding/intro)
* `Mobilizon.Service.Geospatial.Mimirsbrunn` [🔗](https://github.com/CanalTP/mimirsbrunn)
* `Mobilizon.Service.Geospatial.Pelias` [🔗](https://pelias.io)
## Shared options
* `:user_agent` User-Agent string to send to the backend. Defaults to `"Mobilizon"`
* `:user_agent` User-Agent string to send to the backend. Defaults to `"Mobilizon"` or `Mobilizon.Config.instance_user_agent/0`
* `:lang` Lang in which to prefer results. Used as a request parameter or
through an `Accept-Language` HTTP header. Defaults to `"en"`.
* `:country_code` An ISO 3166 country code. String or `nil`
@ -31,7 +33,10 @@ defmodule Mobilizon.Service.Geospatial.Provider do
## Options
Most backends implement all of [the shared options](#module-shared-options).
In addition to [the shared options](#module-shared-options), `c:geocode/3` also
accepts the following options:
* `zoom` Level of detail required for the address. Default: 15
## Examples

View file

@ -33,15 +33,3 @@ theme:
icon: 'calendar_today'
feature:
tabs: true
#
#nav:
# - Home: 'index.md'
# - About: 'about.md'
# - Administration:
# - Install: 'administration/install.md'
# - Dependencies: 'administration/dependencies.md'
# - Docker: 'administration/docker.md'
# - Contribute:
# - Contribute: 'contribute.md'
# - Development: 'contribute/development.md'
# - Styleguide: