Operations/Minutes/2021-09-08
OpenStreetMap Foundation, Operations Meeting* - Agenda & draft minutes
Wednesday 8 September 2021 18:00 London time
Location: Video room at https://osmvideo.cloud68.co
* Please note that this was not strictly an Operations Working Group (OWG) meeting.
Participants
Present:
- Tom Hughes (OWG)
- Grant Slater (OWG)
- Paul Norman (OWG)
- Hrvoje Bogner (OWG)
- Guillaume Rischard (OSMF board)
Apologies:
Minutes by Dorothea Kazazi.
Topics
OSMF email provider
On issue of email bounces
- Emails have been bouncing in >2 past years.
- Random cases (Sarah's emails tend to drop) but not consistently.
- Some genuine Yahoo's emails tend to bounce.
Background
- Foundation is on Google G3 Legacy free tier.
- Settings to control spam are premium options.
- In the past the board was not feeling comfortable paying Google.
- Running an email service is complicated.
- Foundation had decided to move to another provider.
On G3 price
- Once you start paying, you can't go back to the free tier.
- Groups are free,
- $6 per user per month. We have 37 non-suspended users:
- For OSMF it would be ~163 $ / month
Choice: pay or migrate.
Ultimately the Foundation's concern.
In our remit: making sure DMARC is turned-on.
Suggestions:
- Tell the board that OWG not sure if it lies with OWG. Make a decision if switch or pay.
- Guillaume to tell the board that there are ongoing problems with bounces and the OWG will start paying, unless the board objects.
Action items:
- Grant and Guillaume to capture the emails of suspended users and transfer them to an archive account before deleting them and start paying.
- Grant and Guillaume to start paying from the OWG budget, the Treasurer is fine.
Discourse: who should have access to view IP addresses?
People with "Moderator" status in Discourse have the ability to view all IPs.
- Should be mindful of who will be picked as moderator.
- Non-disclosure agreement should be signed by moderators.
Back-ups
- Semi-automatic.
- Use bespoke back-up tool.
- We'll have to write a script to back up.
Action item: Paul to check if Discourse moderators see the back-ups.
Fastly
- Fastly are ok to be live on the "supported-by" list.
- They have sent a draft press release.
Action items:
- Paul to do the PR to add Fastly to the "supported-by" list.
- Grant to send the Fastly draft press release to Paul.
Any other business
Action item: Guillaume to introduce the Junos person from France to the rest.
Back-ups for Cisco configuration in Amsterdam
> Grant might have an ad-hoc one.
> Tom also saved one when they were set-up.
Action item: OWG to create a ticket to create a backup of the Cisco config in Amsterdam.
Render server
Action item: OWG to decide a name on render server(s) in Sweden.
Dublin data center
- Grant ordered 4 sim cards that provide 10 years of connectivity for 10 EUR (500 MB).
- We can connect with an openVPN client.
- Sims work outside of Europe.
- Also ordered two 4G modems.
Action item: OWG to check if Dublin server has been racked (Paul started a wiki page).
Action items
- 2021-09-08 Grant and Guillaume to capture the emails of suspended users and transfer them to an archive account before deleting them and start paying [Topic: OSMF email provider]
- 2021-09-08 Grant and Guillaume to start paying the Google Suite from the OWG budget [Topic: OSMF email provider]
- 2021-09-08 Paul to check if Discourse moderators see the back-ups, [Topic: Discourse]
- 2021-09-08 Paul to do the PR to add Fastly to the "supported-by" list. [Topic: Fastly]
- 2021-09-08 Grant to send the Fastly draft press release to Paul. [Topic: Fastly]
- 2021-09-08 Guillaume to introduce the Junos person from France to the rest. [Topic: AOB]
- 2021-09-08 OWG to create a ticket to create a backup of the Cisco config in Amsterdam. [Topic: AOB Back-ups for Cisco config in Amsterdam]
- 2021-09-08 OWG to decide a name on render server(s) in Sweden. [Topic: Render server]
- 2021-09-08 OWG to check if Dublin server has been racked. (Paul started a wiki page). [Topic: Dublin data center]
2021-08-25 Paul to make a tracking issue for Tilemaker[Topic: Tilemaker] # 2021-09-08 Done- 2021-07-28 OWG to email Fastly that we would like to tweet about them. [Topic: Fastly PR] # 2021-08-25 Guillaume emailed Mikel instead of Michal.
- 2021-07-28 OWG to list Fastly on the website. [Topic: Fastly PR]
- 2021-05-05 Grant to email Toby from WMF and suggest chatting to MapTiler. [Topic: Wikimedia] # 2021-06-02, 2021-06-16, 2021-06-30, 2021-07-14, 2021-08-11, 2021-08-25 Status: Pending. Low priority.
- 2021-03-24 Tom to report back on TimescaleDB again at next meeting. [Topic: Reportage] [was: 2021-01-13 Tom to evaluate TimescaleDB] [Topic: Longer term metric retention] # 2021-04-21 SSD Disk Failing in US # 2021-05-19 decision to leave on the agenda. # 2021-06-02, 2021-06-16, 2021-06-30 nothing new # 2021-07-14 deployed yesterday # 2021-08-11 Failing over again after a week. Hypothesis: issue with Postgres and number of tables.Some of the autovacuum jobs stuck in loops reading the statistics. # 2021-08-25 No update.
- 2021-01-13 OWG to send message to the servers we want to keep. [Reportage. Existing CDN servers] # 2021-03-24 Three servers stopped talking to us (shenron, naga and one more) # 2021-05-19, 2021-06-02 & 2021-06-16 pending #2021-07-14 in progress, half-done. 2021-08-11, 2021-08-25 Pending.
- 2021-01-13 Grant to wipe thorns and the 3 other machines [AMS] [Topic: Longer term metric retention] # 2021-05-19 pending # 2021-06-02 Ramoth data drives wiped - decision: Grant to do final wipe of Ramoth and leave it until next site visit. Discussion about 16GB DDR3. # 2021-07-14 Paul to update ticket with the decision.
- 2020-12-02 Grant to develop some thoughts on what is next for us using AWS. [Topic: AWS] # 2021-05-19 & 2021-06-02 pending, # 2021-06-16 postponed for a few weeks.
- 2020-11-04 OWG to work out tile log archival and deletion policy at later stage. [Topic: Commercial CDN] # 2021-03-24 & 2021-05-19 deferred to future point.
- 2020-10-21 Paul to write to Discourse ticket and email the board [Topic: Discourse] # 2021-06-16 pending.
- 2020-09-09 Grant [Topic: AWS] Speak to AWS person about going ahead with open data program with official OSM S3 bucket. # 2021-05-19 & 2021-06-02 pending.
- 2020-09-09 Grant [Topic: AWS] Talk to OpenAerial Map/HOT. # 2021-05-19 & 2021-06-02 pending. # 2021-06-16 Paul talked to HOT about OpenAerial Map. We can give them geotiff and can produce tiles from it.
- 2020-08-12 Michal to try to rekindle excitement about people helping with imagery (on dev channel/imagery channel or Slack). # 2020-08-26 No progress.
- 2020-07-29 Grant to enable background sync to AWS S3. [Topic: Ironbelly] # 2020-08-12, 2020-08-26 & 2021-06-02 Manually run, automated scripting to be added. # 2021-05-19 Grant to run the script again.
- 2020-07-29 Grant to check with Wiki Admins on hCaptcha (reCaptcha replacement). [Topic: Wiki reCaptcha issue] https://github.com/openstreetmap/operations/issues/454 # 2020-08-12 hCaptcha people reached out and happy to help. Blocker on Mediawiki 1.35 being released in August. # 2021-05-19 blocker removed. # 2021-06-02 pending.
- 2020-07-01 Paul to create a ticket about solutions to reduce incoming comms. [Topic:Revision of acceptable use policy to reduce incoming comms] # 2021-05-19 decision to leave the action item open. # 2021-06-02 discussion about priority for account deletion.
- 2020-07-01 Grant to work out some of the questions for an online form as a solution to reduce incoming comms. [Topic: Revision of acceptable use policy to reduce incoming comms] 2020-08-12 need to think about the reply # 2021-05-19 decision to leave the action item open.
- 2020-06-04 Paul to update the Github ticket "Adding API key support for tile.osm.org" https://github.com/openstreetmap/operations/issues/342
- 2020-04-10 Grant to work out a table of different data bits, work out how they are backed up and what can be potentially improved. [Topic: High Availability / Redundancy of OpenStreetMap.org (and primary services)] # 2021-05-19 decision to leave the action item open. # 2021-06-02 pending.
Next meeting
Wednesday 22 September 2021 18:00 London time
Operations meetings are currently being held every 2 Wednesdays, at 18:00 London time.
Online calendar showing the OPS meetings.