OpenStreetMap Foundation, Operations Meeting* - Agenda & draft minutes
Wednesday 25 August 2021 18:00 London time
Location: Video room at https://osmvideo.cloud68.co
* Please note that this was not strictly an Operations Working Group (OWG) meeting.
- Tom Hughes (OWG)
- Grant Slater (OWG)
- Paul Norman (OWG)
- Guillaume Rischard (OSMF board)
- Tobias Knerr (OSMF board)
- Richard Fairhurst (Guest - Tilemaker topic)
Minutes by Dorothea Kazazi.
Would it make sense for the OSMF to host a set of vector tile generated by tilemaker?
https://github.com/openstreetmap/operations/issues/565 (created after the meeting)
On one of the donated server, probably after we got Sweden and Dublin running.
Requirements for planet generation
- 128-160 GB RAM
- SSD for workspace
- time: in a couple of days on 16 cores (on 5650 Intel Xeon)
Generation of vector-tiles up to zoom level 14, in open maptiles schema.
Generation in standard Mapbox vector tiles format MVT -> SQlite container, can be rendered by any MVT-rendering library (Mapbox GL JS, maplibre-gl-js, Mapzen's Tangram).
- new ordered machines: 256 G (probably won't use those)
- necrosan: 196 GB, 12 cores
- scorch: 128 GB, 8 cores
- rhaegal in Croatia: 256 GB, 24 cores
use HPDL3.60 GEN8?
Vector tiles is a massive project.
- Encouraging dev of osm.org
- Fun thing to get started with.
- Proof of concept.
- Not a production service or a long-term solution.
Do a rendering example straight out of the box? > Yes, simple and will get people thinking.
Storage capacity for mbtiles
Can it be done similar with cycle travel, in parallel by continent?
> Yes - use of osmium to split the planet.
- Initial weekly interval for updates of vector tiles.
- Try to get an example web stack up before announcement.
- Put the tiles through Fastly with current rate limiting.
- Start with a single backend.
On making mbtiles available
Single file/container/SQL database that contains binary blobs.
- Put container on planet server? > Depends on storage space.
- Explode the web-request URL path, it gets written in Ruby script that fetches individual tiles.
On potential legal issues
> Default style: if using openmaptiles style - attribution.
Tilemaker has compatible version of schema.
Action item: Paul to make a tracking issue.
Amanda has been hosting Mastodon.
Board vote: 2021/Res29 OSMF takes over funding for the en.osm.town Mastodon instance/server
- Could fit with OWG budget.
- Aligns with free software goals.
- Disadvantage: Another thing to run.
Keep it as it is for now. Finish current community website and then decide. Could then put it on OWG budget and enable OAuth 2.0 to it.
- We know what plugins we want to use.
- No simple way to roll it progressively.
- mmd and Andy tested Discourse.
Suggestion: In future look at ways for progressive roll outs.
Tom to merge the open ticket.
- can be checked to not match osm.org. Can be changed back when relog in osm.org?
- Username and email can be automatically forced.
IP addresses appearing in admin panel of Discourse are wrong - possibly due to Shorewall.
- Private messages on osm.org, Discourse and forum.
- Some forum users want the forum content to be migrated.
- It has been successfully done by others.
- Fedora decided not to migrate, when switched to Discourse.
- Migration of mailing lists' content probably a no.
- Forum migration possible. Do we want to? > Not yet. Let the community decide.
- Can keep forum on read-only status.
Timeline: probably mid-September will go live.
How will categories will look - non-EN communities? (Question by Tobias)
> Suggestion to talk to Rubén Martín on Slack.
> Decisions about categories will be left to other people.
Communication about the launch
- OWG will come up with text and run it by Communication Working Group
- Some communications might happen sooner on Twitter.
2021-08-11 Grant to look at the DAC cables to order from German supplier. [Topic: New render servers] 2021-08-11 Guillaume to redirect parked/broken domains to openstreetmap.org [Topic: Redirect parked/broken domains to openstreetmap.org] 2021-08-11 Grant and Paul to sync and one of them to be present during the August 27 public board meeting to give an update about Dublin data center. [Topic: Update about Dublin data center during the public board meeting]
- 2021-07-28 OWG to email Fastly that we would like to tweet about them. [Topic: Fastly PR] # 2021-08-25 Guillaume emailed Mikel instead of Michal.
- 2021-07-28 OWG to list Fastly on the website. [Topic: Fastly PR]
- 2021-05-05 Grant to email Toby from WMF and suggest chatting to MapTiler. [Topic: Wikimedia] # 2021-06-02, 2021-06-16, 2021-06-30, 2021-07-14, 2021-08-11, 2021-08-25 Status: Pending. Low priority.
- 2021-03-24 Tom to report back on TimescaleDB again at next meeting. [Topic: Reportage] [was: 2021-01-13 Tom to evaluate TimescaleDB] [Topic: Longer term metric retention] # 2021-04-21 SSD Disk Failing in US # 2021-05-19 decision to leave on the agenda. # 2021-06-02, 2021-06-16, 2021-06-30 nothing new #2021-07-14 deployed yesterday # 2021-08-11 Failing over again after a week. Hypothesis: issue with Postgres and number of tables.Some of the autovacuum jobs stuck in loops reading the statistics. # 2021-08-25 No update.
- 2021-01-13 OWG to send message to the servers we want to keep. [Reportage. Existing CDN servers] # 2021-03-24 Three servers stopped talking to us (shenron, naga and one more) # 2021-05-19, 2021-06-02 & 2021-06-16 pending #2021-07-14 in progress, half-done. 2021-08-11, 2021-08-25 Pending.
- 2021-01-13 Grant to wipe thorns and the 3 other machines [AMS] [Topic: Longer term metric retention] # 2021-05-19 pending # 2021-06-02 Ramoth data drives wiped - decision: Grant to do final wipe of Ramoth and leave it until next site visit. Discussion about 16GB DDR3. #2021-07-14 Paul to update ticket with the decision.
- 2020-12-02 Grant to develop some thoughts on what is next for us using AWS. [Topic: AWS] # 2021-05-19 & 2021-06-02 pending, 2021-06-16 postponed for a few weeks.
- 2020-11-04 OWG to work out tile log archival and deletion policy at later stage. [Topic: Commercial CDN] # 2021-03-24 & 2021-05-19 deferred to future point.
- 2020-10-21 Paul to write to Discourse ticket and email the board [Topic: Discourse] # 2021-06-16 pending.
- 2020-09-09 Grant [Topic: AWS] Speak to AWS person about going ahead with open data program with official OSM S3 bucket. # 2021-05-19 & 2021-06-02 pending.
- 2020-09-09 Grant [Topic: AWS] Talk to OpenAerial Map/HOT. # 2021-05-19 & 2021-06-02 pending. # 2021-06-16 Paul talked to HOT about OpenAerial Map. We can give them geotiff and can produce tiles from it.
- 2020-08-12 Michal to try to rekindle excitement about people helping with imagery (on dev channel/imagery channel or Slack). # 2020-08-26 No progress.
- 2020-07-29 Grant to enable background sync to AWS S3. [Topic: Ironbelly] # 2020-08-12, 2020-08-26 & 2021-06-02 Manually run, automated scripting to be added. # 2021-05-19 Grant to run the script again.
- 2020-07-29 Grant to check with Wiki Admins on hCaptcha (reCaptcha replacement). [Topic: Wiki reCaptcha issue] https://github.com/openstreetmap/operations/issues/454 # 2020-08-12 hCaptcha people reached out and happy to help. Blocker on Mediawiki 1.35 being released in August. # 2021-05-19 blocker removed. # 2021-06-02 pending.
- 2020-07-01 Paul to create a ticket about solutions to reduce incoming comms. [Topic:Revision of acceptable use policy to reduce incoming comms] # 2021-05-19 decision to leave the action item open. # 2021-06-02 discussion about priority for account deletion.
- 2020-07-01 Grant to work out some of the questions for an online form as a solution to reduce incoming comms. [Topic: Revision of acceptable use policy to reduce incoming comms] 2020-08-12 need to think about the reply # 2021-05-19 decision to leave the action item open.
- 2020-06-04 Paul to update the Github ticket "Adding API key support for tile.osm.org" https://github.com/openstreetmap/operations/issues/342
- 2020-04-10 Grant to work out a table of different data bits, work out how they are backed up and what can be potentially improved. [Topic: High Availability / Redundancy of OpenStreetMap.org (and primary services)] # 2021-05-19 decision to leave the action item open. # 2021-06-02 pending.
Operations meetings are currently being held every 2 Wednesdays, at 18:00 London time.
Online calendar showing the OPS meetings.