Jump to content

Operations/Minutes/2026-08-06

From OpenStreetMap Foundation

OpenStreetMap Foundation, Operations Meeting - Draft minutes

These minutes do not go through a formal acceptance process.
This is not strictly an Operations Working Group (OWG) meeting.

Thursday 6 August 2026, 19:00 London time
Location: Video room at https://osmvideo.cloud68.co

Participants

Minutes by Dorothea Kazazi.

New action items from this meeting

  • Tom to update the notes on how to perform Postgres version upgrades. For major Postgres versions we have to run stanza-upgrade on the repository before we start up the new version of Postgres. Does not apply for point releases.[Topic: pgBackRest upgrades]
  • OPS to do test on one of the slave servers and as part of that test a pgBackRest restore. [Topic: pgBackRest upgrades]
  • OPS to decide whether to enable page checksums. This will be decoupled from the pgBackRest upgrade, as it requires significant downtime. [Topic: pgBackRest upgrades]
  • Rubén to do a storage comparison experiment using traces of various sizes, load them into a database in the current schema and as LineStrings, and compare table sizes. [Topic: GPS points database table]

Dublin data center site visit

Related to https://github.com/openstreetmap/operations/issues?q=is%3Aissue%20state%3Aopen%20label%3Alocation%3Adublin

We still need a name for the QGIS sponsored tile server.

Grant will travel to the Dublin data center on Saturday for infrastructure maintenance.

Tasks

  • Change the out-of-band Raspberry Pi to match the Amsterdam one.
  • Install the new power switching device for the Raspberry Pis.
  • De-rack two old servers.
  • Rack the new QGIS-sponsored tile render server.
  • Replace the 4G modem antenna with a significantly better one and reposition it.
  • Replace an interlink cable between the two PDUs with a lower quality one. The replacement will not be a Cat6 but likely an aluminum Ethernet cable.
  • Replace a memory DIMM in Smaug (Nominatim machine).

pgBackRest upgrades

Backups

  • Scheduled backup now runs on a Thursday to avoid existing pg_backup
  • The first automated full backup is currently running and is 46% complete. It should finish around lunchtime tomorrow. It is expected to take approximately 36 hours.Subsequent backups should take less than 12 hours.

There is a new minor version pgBackRest upgrade available. We are running the version from the Postgres repository, not the Debian one.

For restoration The Postgres version has to match for restoration, otherwise replacement of the WAL log files throws an error.

Suggestions

  • Enabling page checksums (a Postgres 18 feature).
    • This will require tens of hours of downtime.

Action items

  • Tom to update the notes on how to perform Postgres version upgrades. For major Postgres versions we have to run stanza-upgrade on the repository before we start up the new version of Postgres. Does not apply for point releases.
  • OPS to do test on one of the slave servers and as part of that test a pgBackRest restore.
  • OPS to decide whether to enable page checksums. This will be decoupled from the pgBackRest upgrade, as it requires significant downtime.

GPS points database table

As part of the GPX modernization, is there space in the server/database for a table where gps_points could be stored as a LineString?

Issue: GPS trace pagination is currently blocked because GPS points do not have a primary key.

Suggestions regarding absence of primary key

  • Use the GPX file ID or track ID plus timestamp.
  • Use the GPX file ID, possibly plus the track ID, as the cursor and not split pages inside a track. Thus, there would be no concern of using the timestamp in the cursor. Each page would contain either a set number of complete tracks, or roughly a target number of points, always cutting at a track boundary rather than mid-track.

On suggestion to convert the GPS points into LineStrings
Suggestion: convert the GPS points into LineString geometry stored in a new table, to enable pagination and to prepare for future tiling work.

  • It is unclear whether a LineString / multi-Linestrings or point collection is preferable.
  • Converting GPS points to LineStrings does not actually make the pagination problem any easier.

Suggestions

  • Sketch out other things we want to do with LineStrings, e.g. clipping for privacy reasons.
  • Look at the storage implications and compare the space to store a given trace using LineStrings versus the current schema.

Action item

Rubén to do a storage comparison experiment using traces of various sizes, load them into a database in the current schema and as LineStrings, and compare table sizes.


Action items

  • 2026-07-23 Minh to mention again the GPS trace visibility change in the weekly recap, outlining the privacy implications. [Topic: GPS trace visibility change]
  • 2026-07-23 Minh to follow up with mmd. [Topic: cgimap maintenance]
  • 2026-07-23 Grant and Minh to discuss the MediaWiki OAuth extension work. [Topic: cgimap maintenance]
  • 2026-07-23 Grant to proceed with the MediaWiki upgrade on the test wiki in the next few days. [Topic: MediaWiki upgrade]
  • 2026-07-23 Minh to do a cursory testing of extensions on the test wiki, once it is upgraded. [Topic: MediaWiki upgrade]
  • 2026-07-23 Grant to look into the Fastly timeout settings (connection timeout, first-byte timeout, and between-bytes timeout) and attempt to reproduce the GPX time-out issue using developer tools. [Topic: GPX uploads fail with 503]
  • 2026-05-28 Minh to follow up on DWG non-disclosure agreements, then IP access for moderators in website repo. [DWG request for user info for moderators]
  • 2026-05-28 Paul to ask OSM France how do they view if we retire the HOT style from osm.org since it is not under active development [AOB: Tracestrack topo and HOT layers]
  • 2026-05-14 Grant to check with Fastly team via Slack about 100% ngwaf traffic
  • 2026-05-14 Paul to do PR setting referer policy header on embed — Live
  • 2026-03-19 Grant to research what triggers a large download from QGIS. [Topic: QGIS Tiles usage] # 2026-05-14 Not much progress; unable to find the new usage header in latest versions. # 2026-06-11 Action item to be parked.
  • 2026-03-19 Paul and Grant to run some time limited experiments during non peak hours to test catching anonymous/fake-ua scrapers. Genuine Google Bot etc will continue to be permitted. [Topic: Fastly Client Challenges]
  • 2025-10-16 Grant and Paul to set up a meeting about AWS Identity and Access Management Roles Anywhere https://docs.aws.amazon.com/rolesanywhere/latest/userguide/introduction.html. [Topic: AWS CA cert] # 2026-05-14 Grant to chat to Tom about what CA + Signed Certs are required. Grant to read the AWS documentation on requirements.
  • 2025-10-16 Grant to create a PR regarding refactoring some stuff. Make kitchen handle selecting cookbook-OS combos. and only running tests on cookbooks that have changed [Topic: Reworking of Test Kitchen methods for defining which jobs run on Test Kitchen GitHub actions]
  • 2025-10-16 Grant to create a PR about adding logic to Chef for retrying failed initial creation of Let's Encrypt certificates. [Topic: Add logic to Chef for retrying failed initial creation of Let's Encrypt certificates]