Operations/Minutes/2025-01-09
OpenStreetMap Foundation, Operations Meeting - Draft minutes
These minutes do not go through a formal acceptance process.
This is not strictly an Operations Working Group (OWG) meeting.
Thursday 09 January 2025, 19:00 London time
Location: Video room at https://osmvideo.cloud68.co
Participants
- Tom Hughes (OWG volunteer)
- Grant Slater (SSRE, OWG)
Minutes by Dorothea Kazazi.
Absent
- Paul Norman (OWG volunteer)
Keep HE.net around?
Also related to the AOB topic "2024-12 outage".
On suggestion to keep HE.net around as a backup
- Manual IP changes needed during failover, so not an ideal failover provider.
- There are things we could do about it.
Other points mentioned during discussion
- OSMF has been saying that there is budget availability for redundancy measures.
- Equinix internet: is free and has more redundant configuration with two routers.
Concensus seemed to be: Keep HE.net. No opposition to keeping HE.net around was stated.
Karlsruhe Hack Weekend
There is going to be an OSM hack weekend in Karlsruhe, Germany, on 15 and 16 February 2025.
People who will be there include: Grant, Andy Allan, Sarah Hoffmann, and Florian Lohoff, who has worked with large ISPs and had offered support to the OPS.
- Grant would like to talk to Florian about our networking and uplink redundancy.
- Grant was told that the OSMF will cover Tom's expenses, if Tom wanted to join.
Suggestions
- Grant suggested that it would be nice for Tom to join.
- Grant to confirm with the board regarding OSMF covering Tom's expenses.
Other points mentioned during discussion
- Tom is considering going, if Andy will be there.
- The attendee list is limited, so Tom would have to add himself to the OSM wiki list.
- Last year, there were 23 attendees and the event was moved to the university.
Any other business
Lockheed issues
- Rebouted 5-6 times last week.
- Grant is thinking of going to Amsterdam on the return trip from Karlsruhe.
- Plan: Swap out the network card, probably the controller and upgrade the RAM.
Outage post-mortem
Grant has been working on the post-mortem about the osm.org and API outage on 15-17 December 2024.
Report includes:
- Timeline of events
- Assessment of impact
- Root cause analysis
- Short-term steps
- Recommendations for improved resilience
- Actions
Action items
- Grant to circulate the draft of the outage post-mortem privately in the operations channel and to the board for feedback.
- Grant to check whether the links dropped just before the restore.
Next meeting
2025-01-23
Action items reviewed at the beginning of the meeting
- 2024-11-14 Paul to add the cache control headers [Topic: budget]
- 2024-10-31 Paul to contact OMT about their application to be added as a featured layer on www.osm.org. [Topic: OpenMapTiles application]
- 2024-10-31 Paul to look where OMT pulls their fonts from and their style file. [Topic: OpenMapTiles application]
- 2024-10-31 Paul to contact OMT about their application to be added as a featured layer on www.osm.org. [Topic: OpenMapTiles application]
- 2024-10-31 Paul to look where OMT pulls their fonts from and their style file. [Topic: OpenMapTiles application]
- 2024--09-19 Grant to create a ticket for action item 2024-08-08 OPS to evaluate Fastly Security (DDOS) Protections we could use. [Topic: Cloudflare / Fastly]
- 2024--09-19 Grant to create an IP blocklist. [Topic: Cloudflare keep enabled?][2024-09-19 Reportage] - Discussion during 2024-07-25 OPS to make a reasonable evaluation whether to go with Cloudflare, Fastly or none.
- 2024--09-19 Grant to confirm that the AArnet servers will be removed and to ask the Australian community whether there is interest in hosting/providing a render server in Australia or Asia/Pacific [2024-09-19 topic: AArnet Servers going away]
- 2024-08-22 Guillaume to make a limited in scale experiment to assess impact and practicality. E.g. look if there are clients that say they support WEBP and don't. [Topic: Fastly image recoding]
- 2024-08-22 Guillaume to keep OPS in the loop about what Fastly says. [Topic: Fastly image recoding]
- 2024-08-22 Grant to talk to Guillaume on setting up the testing about image recoding and shielding. [Topic: Fastly image recoding]
- 2024-07-25 OPS to make a reasonable evaluation whether to go with Cloudflare, Fastly or none. [Topic: Cloudflare keep enabled?]
- 2024-06-27 OPS to do capacity planning for tile.openstreetmap.org [Topic: rhaegel usage?]
Action items that have been stricken-through are completed, removed, or have been moved to GitHub tickets.