Operations/Minutes/2021-09-22
OpenStreetMap Foundation, Operations Meeting* - Agenda & draft minutes
Wednesday 22 September 2021 18:00 London time
Location: Video room at https://osmvideo.cloud68.co
* Please note that this was not strictly an Operations Working Group (OWG) meeting.
Participants
Present:
- Tom Hughes (OWG)
- Grant Slater (OWG)
- Paul Norman (OWG)
- Hrvoje Bogner (OWG)
- Michal Migurski
- Guillaume Rischard (OSMF board)
Apologies:
Minutes by Dorothea Kazazi.
Topics
Dublin data center
Juniper configurations
Plan: Create a draft document of what we want to do.
What we want
- Link aggregation on Gigabit ports. VLAN running on top of those.
- Tagged traffic and one of the tags to get stripped when the traffic reaches the port. Bond the network cards, run tags over those, except for the internal one, which is untagged. Once it gets to the switches, it becomes tagged.
Documentation seems to be ad-hoc regarding which model they refer to.
- We run 2 logical networks over 1 physical infrastructure. Public network is tagged with VLAN2. Untagged traffic has private IP addresses.
- Can filter on IP addresses but that's not how we did it on Amsterdam.
Action items:
- Grant to create a draft document of what we want to do.
- Grant to figure the Juniper configuration with Guillaume and the French person.
Pending arrivals
- API database server
- Planet storage server
Sent wrong receipt date to Equinix for one inbound shipment. Need to confirm receipt.
Action item: Grant to phone Equinix.
Preparations for Dublin visit
- Shelf
- Spare PSUs
- Thermosensors for APCs
- Cables: power cables, ethernet cables
- Cable ties/zip ties
- Spare disks
- Don't have - labelling machine
Racks are deep so cables might be short.
Suggestion: contact Dermot McNally.
Fastly
Fastly now appears on www.osm.org
To do:
- Send a tweet from the official OSM account.
- Technical blog post.
- Fastly press release.
Action item: Paul to check the draft post sent by Fastly for accuracy.
Paul will move to Amazon location service team in AWS (in October at Vancouver).
Action items
- 2021-09-08 Grant and Guillaume to capture the emails of suspended users and transfer them to an archive account before deleting them and start paying [Topic: OSMF email provider] # 2021-09-22 Pending. Guillaume may have cleared some.
- 2021-09-08 Grant and Guillaume to start paying the Google Suite from the OWG budget [Topic: OSMF email provider] # 2021-09-22 Pending.
2021-09-08 Paul to check if Discourse moderators see the back-ups. [Topic: Discourse]- 2021-09-08 Paul to do the PR to add Fastly to the "supported-by" list. [Topic: Fastly] # 2021-09-22 Added in one place. Probably still needs to be added to hardware.openstreetmap.org
- 2021-09-08 Grant to send the Fastly draft press release to Paul. [Topic: Fastly] # 2021-09-22 Pending.
- 2021-09-08 Guillaume to introduce the Junos person from France to the rest. [Topic: AOB]
- 2021-09-08 OWG to create a ticket to create a backup of the Cisco config in Amsterdam. [Topic: AOB Back-ups for Cisco config in Amsterdam] # 2021-09-22 Pending. Ticket not made but backup has.
2021-09-08 OWG to decide a name on render server(s) in Sweden. [Topic: Render server]2021-09-08 OWG to check if Dublin server has been racked. (Paul started a wiki page). [Topic: Dublin data center]- 2021-07-28 OWG to email Fastly that we would like to tweet about them. [Topic: Fastly PR] # 2021-08-25 Guillaume emailed Mikel instead of Michal. # 2021-09-22 We received a couple of emails by them.
2021-07-28 OWG to list Fastly on the website. [Topic: Fastly PR]- 2021-05-05 Grant to email Toby from WMF and suggest chatting to MapTiler. [Topic: Wikimedia] # 2021-06-02, 2021-06-16, 2021-06-30, 2021-07-14, 2021-08-11, 2021-08-25 Status: Pending. Low priority.
- 2021-03-24 Tom to report back on TimescaleDB again at next meeting. [Topic: Reportage] [was: 2021-01-13 Tom to evaluate TimescaleDB] [Topic: Longer term metric retention] # 2021-04-21 SSD Disk Failing in US # 2021-05-19 decision to leave on the agenda. # 2021-06-02, 2021-06-16, 2021-06-30 nothing new # 2021-07-14 deployed yesterday # 2021-08-11 Failing over again after a week. Hypothesis: issue with Postgres and number of tables.Some of the autovacuum jobs stuck in loops reading the statistics. # 2021-08-25 No update.
- 2021-01-13 OWG to send message to the servers we want to keep. [Reportage. Existing CDN servers] # 2021-03-24 Three servers stopped talking to us (shenron, naga and one more) # 2021-05-19, 2021-06-02 & 2021-06-16 pending #2021-07-14 in progress, half-done. 2021-08-11, 2021-08-25 Pending.
- 2021-01-13 Grant to wipe thorns and the 3 other machines [AMS] [Topic: Longer term metric retention] # 2021-05-19 pending # 2021-06-02 Ramoth data drives wiped - decision: Grant to do final wipe of Ramoth and leave it until next site visit. Discussion about 16GB DDR3. # 2021-07-14 Paul to update ticket with the decision.
- 2020-12-02 Grant to develop some thoughts on what is next for us using AWS. [Topic: AWS] # 2021-05-19 & 2021-06-02 pending, # 2021-06-16 postponed for a few weeks.
- 2020-11-04 OWG to work out tile log archival and deletion policy at later stage. [Topic: Commercial CDN] # 2021-03-24 & 2021-05-19 deferred to future point.
- 2020-10-21 Paul to write to Discourse ticket and email the board [Topic: Discourse] # 2021-06-16 pending.
- 2020-09-09 Grant [Topic: AWS] Speak to AWS person about going ahead with open data program with official OSM S3 bucket. # 2021-05-19 & 2021-06-02 pending.
- 2020-09-09 Grant [Topic: AWS] Talk to OpenAerial Map/HOT. # 2021-05-19 & 2021-06-02 pending. # 2021-06-16 Paul talked to HOT about OpenAerial Map. We can give them geotiff and can produce tiles from it.
- 2020-08-12 Michal to try to rekindle excitement about people helping with imagery (on dev channel/imagery channel or Slack). # 2020-08-26 No progress.
- 2020-07-29 Grant to enable background sync to AWS S3. [Topic: Ironbelly] # 2020-08-12, 2020-08-26 & 2021-06-02 Manually run, automated scripting to be added. # 2021-05-19 Grant to run the script again.
- 2020-07-29 Grant to check with Wiki Admins on hCaptcha (reCaptcha replacement). [Topic: Wiki reCaptcha issue] https://github.com/openstreetmap/operations/issues/454 # 2020-08-12 hCaptcha people reached out and happy to help. Blocker on Mediawiki 1.35 being released in August. # 2021-05-19 blocker removed. # 2021-06-02 pending.
- 2020-07-01 Paul to create a ticket about solutions to reduce incoming comms. [Topic:Revision of acceptable use policy to reduce incoming comms] # 2021-05-19 decision to leave the action item open. # 2021-06-02 discussion about priority for account deletion.
- 2020-07-01 Grant to work out some of the questions for an online form as a solution to reduce incoming comms. [Topic: Revision of acceptable use policy to reduce incoming comms] 2020-08-12 need to think about the reply # 2021-05-19 decision to leave the action item open.
- 2020-06-04 Paul to update the Github ticket "Adding API key support for tile.osm.org" https://github.com/openstreetmap/operations/issues/342
- 2020-04-10 Grant to work out a table of different data bits, work out how they are backed up and what can be potentially improved. [Topic: High Availability / Redundancy of OpenStreetMap.org (and primary services)] # 2021-05-19 decision to leave the action item open. # 2021-06-02 pending.
Next meeting
Wednesday 6 October 2021 18:00 London time
Operations meetings are currently being held every 2 Wednesdays, at 18:00 London time.
Online calendar showing the OPS meetings.