Operations/Minutes/2025-05-01

From OpenStreetMap Foundation

OpenStreetMap Foundation, Operations Meeting - Draft minutes

These minutes do not go through a formal acceptance process.
This is not strictly an Operations Working Group (OWG) meeting.

Thursday 1 May 2025, 19:00 London time
Location: Video room at https://osmvideo.cloud68.co

Participants

Minutes by Dorothea Kazazi, including notes by Grant.

Absent

New action items from this meeting

  • Grant to follow-up with Australian hosting again. [Topic: OSUOSL funding / issues]
  • Grant to see if other University offers are still available and what hardware would be required. [Topic: OSUOSL funding / issues]
  • Grant to check with Ian if he would be willing to store the hardware if the worst outcome happened. [Topic: OSUOSL funding / issues]
  • Grant to run past Paul on the separation of AWS credit applications. [Topic: AWS Credits application]
  • Minh to confirm if the proposed wiki extension installation is definitely needed. [Topic: Wiki extension installation requests]
  • Tom to clarify which OSM wiki extension between DynamicPageList and DynamicPageListEngine is better or compatible. [Topic: Wiki extension installation requests]
  • Grant to test OpenTofu upgrades and communicate via IRC. [Topic: OpenTofu upgrades]

Reportage

2025 OWG budget

Craig Allan (Board) has proposed giving to OWG extra 10K. Part of the amount could be provided to OSUOSL (see below).

OSUOSL funding / issues

Oregon State University Open Source Lab (OSUOSL) has been hosting 3 OSMF machines in North America for a long time. OSUOSL have financial problems: they have to get ~ USD 150,000 by the middle of May 2025. They put out a plea for financial support: https://osuosl.org/blog/osl-future/ They seem to have received about half of that amount.

Pending board funding availability and approval we should be in favour of making a tokenary donation to OSUOSL.

In case OSUOSL can no longer host the 3 OSMF machines due to not meeting their budget

  • Nominatim and Tile render would be badly affected if OSUOSL went away. It would be ideal if we had something in North America.
  • Australia and Southeast Asia would be negatively affected, as they go through the US.
  • All the tiles would go to Europe.
  • We would have to ship the OSMF machines to the next hosting provider or to someone in North America. Prometheus is fine moving.
  • We don't' know what the shutdown timeline would be. They would probably provide sufficient notice.

Minh joined ~9 minutes after start.

Suggestion

  • Grant to check the log of the OSUOSL IRC channel.

Action items

  • Grant to follow-up with Australian hosting again.
  • Grant to see if other University offers are still available and what hardware would be required.
  • Grant to check with Ian if he would be willing to store the hardware if the worst outcome happened.

Prometheus to Debian 12

  • We will test the cookbook.
  • We need to check tile style people if any schema changes are pending before a reload. Who needs to be confirmed. Ticket on stylesheet.

AWS Credits application

We currently have two render servers: one at OSU and an AWS one.

The application deadline for AWS credits is May 9th, 2025. Current AWS credits are applied to cover both the cost of the AWS render server (~$300-400/month - mostly for bandwidth) and other OSM services (S3 backups etc, S3 user images and gpx files). If we don't get the AWS credits, we will turn off the AWS render service, as it's too expensive to maintain.

Suggestions

  • Submit two applications for AWS credits this year 1) tile render 2) other OSM AWS usage (S3 backups etc, S3 user images and gpx files). Suggestion by Paul.
  • Move the traffic off the AWS server during low usage times, to save many of the AWS credits.

Other points mentioned during discussion

  • The AWS open data program funds planet.osm.org and it is not billed to us.
  • We have all the AWS budgeting tools turned on, so we can produce the data needed to determine the credits we would require for the other (non render server related) OSM services.

Action item: Grant to run past Paul on the separation of AWS credit applications.

Wiki extension installation requests

There are two tickets, each requesting a different OSM wiki extension, they both do essentially the same thing and they both seem minimally maintained.

On OSM wiki extension request: DynamicPageList hhttps://github.com/openstreetmap/operations/issues/1126

  • This extension was originally requested because Wikimedia is using it.
  • Wikimedia people have provided negative feedback about this extension. This might be because it is used on some wikis with very big categories and they might have concerns about scalability.
  • Minh: Request is low priority.
  • The extension isn't loading on the test OSM wiki, needs debugging.

Link shared during the meeting: https://github.com/openstreetmap/operations/issues/1127#issuecomment-2845440143

Suggestions

  • Enable stack trace dumping to identify the specific error resulting in the extension not loading on the test OSM wiki.
  • Test installations of the other requested extensions.

Other points mentioned during discussion

  • There is much enthusiasm about one of the extensions.
  • The OSM wiki users do not care which of the two extensions will get installed.

Action items

  • Minh to confirm if the proposed wiki extension installation is definitely needed.
  • Tom to clarify which OSM wiki extension between DynamicPageList and DynamicPageListEngine is better or compatible.

OpenTofu upgrades

Grant has set of set of OpenTofu updates pending

If we don't keep OpenTofu up to date, it then becomes a massive upgrade later on. We updated the module around a year ago. While we need to upgrade the OpenTofu version, there should be no changes.

  • Manages: All of our AWS S3 buckets, roles and tokens, the replication of the buckets, public planet.
  • It can manage all the permissions for the replications of the buckets.

- StatusCake module: Needs updating.
- PagerDuty module: Needs updating.
- Fastly module: Is minimal. It manages one data set and runs multiple times daily, updating the rules for the blocked tile sites.

Other point mentioned during discussion

  • When Grant set-up the AWS, he backported all the manual changes into it.

Terraform-Azure

Link shared: https://github.com/openstreetmap/terraform-azure/

Managing translation-related stuff.

Suggestion: Talk to Microsoft to get credits.

Action item: Grant to test upgrades and communicate via IRC.

Open Ops Tickets

Review open, what needs policy and what needs someone to help with...

Next meeting

  • On 2025-05-15.

Action items reviewed at the beginning of the meeting

  • 2025-04-17 Minh Nguyễn to post about the OSM wiki downtime planned for Saturday 26 April 2025 at 10am (9am UTC). [Topic: Wiki upgrades next steps] Done.
  • 2025-04-17 Grant Slater to do a quick compatibility check for the Prometheus server upgrade. [Topic: Next Debian Upgrades?]
  • 2025-03-20 Grant to investigate whether Karm's latency spike on 10 Jan 2025 is due to IO or network. Most likely IO. Karm may need upgrading to handle sync. [Topic: Database Server Upgrades]
  • 2025-03-20 Grant to set the sys request variable to be more dynamic, as we tune the number of threads that MDRAID enables, and it is likely not more than four. [On 10 Jan 2025 peak] [Topic: Database Server Upgrades] Done.
  • 2025-03-20 Grant to negotiate with HE.net if we can get better cost from them as a fallback link (which he had proposed), to allow budget spend elsewhere. [Topic: HE.net]
  • 2025-03-20 Grant to follow-up with the South African contact about the potential hardware donation from a mobile network. [Topic: New offers of servers in Australia and South Africa]
  • 2025-03-20 Grant to run an SQL query to identify more email providers used by spammers. [Topic: Spam]
  • 2025-03-20 Grant to check the metrics for any significant impact of recent spam blocking. [Topic: Spam] Done. 10% drop-off.
  • 2025-03-06 Grant to present a draft budget at the next meeting.
  • 2025-01-23 Grant to check whether Paul wants to pick up responding to Meta [Topic: Rapid editor] In board hands with Mikel / Paul.
  • 2024-09-19 Grant to create an IP blocklist script. [Topic: Cloudflare keep enabled?][2024-09-19 Reportage] - Discussion during 2024-07-25 OPS to make a reasonable evaluation whether to go with Cloudflare, Fastly or none. - Grant to create now
  • 2024-09-19 Grant to confirm that the AArnet servers will be removed and to ask the Australian community whether there is interest in hosting/providing a render server in Australia or Asia/Pacific [2024-09-19 topic: AArnet Servers going away] - We in conversation with Australian National University for new hardware.

Action items that have been stricken-through are completed, removed, or have been moved to GitHub tickets.

Retrieved from "https://osmfoundation.org/w/index.php?title=Operations/Minutes/2025-05-01&oldid=13582"