Operations/Minutes/2024-05-30

From OpenStreetMap Foundation

OpenStreetMap Foundation, Operations Meeting - Draft minutes

These minutes do not go through a formal acceptance process.
This is not strictly an Operations Working Group (OWG) meeting.

Thursday 30 May, 19:00 London time
Location: Video room at https://osmvideo.cloud68.co

Participants

Not present

Minutes by Dorothea Kazazi.

New action items from this meeting

  • Grant to revert the request timeouts. [Topic: Discussion of false positives hitting DDOS protections due to request timeouts hitting max allowed and DDOS protection.]
  • Tom to amend the draft editor policy document and move it from Gdoc to hack.md [Topic: Editor Policy]
  • OPS to add the SDRP requirement to the Editor Policy draft and see what feedback we receive. [Topic: Editor Policy]
  • Grant to download a back-up of Fastly's VCL, every month. [Topic: Fastly future projects]

Reportage

UCL visit

Official process for taking goods out is via the goods in/goods out. A form has to be filled and the requirements are:

  • there needs to be a ticket for it.
  • only allowed to do it during business hours.

Grant has access to the ticketing system and can create a support ticket, which is probably different from the type of ticket that UCL needs. Grant will follow up with a person from UCL.

Discussion of false positives hitting DDOS protections due to request timeouts hitting max allowed and DDOS protection.

https://github.com/openstreetmap/chef/commit/ab4b82d3ca1eb1be198b331203f3505c4a043b59

Issue affecting few different people, including people in Bloomberg.

Options

  • Be more lenient on negotation time.
  • Require more of them before we trigger the block,
  • Both.

Changes by Tom

  • December: fail2ban block on the timeouts. This was implemented because all the Apache slots were tied up by connections that weren't doing anything.
  • January: reduced timeouts.

Tom's changes of negotation time resulted in peaks during the day to be significantly higher. There are also more peaks during weekdays than at weekends. The body was not changed, as people uploading GPS traces might take some time.

Other points mentioned

  • Apache provides only absolute timeouts, while what we need is the timeout since it last sent any data.
  • We can extend the maximum time but keep the minimum time the same.
  • Reduced time-outs in January.

Suggestions

  • Revert the changes.
  • Also change the number of incidents at the same time.
    • It might be better to do one change at a time.

Decisions

  • We will revert the timeouts
  • Will not modify fail2ban until we have feedback from the change.

Action item

Grant to revert the request timeouts.

Editor Policy

- We will add a SDRP requirement to the Editor Policy draft and see what feedback we receive.

Related to action item: 2024-05-16 Paul to pull the proposed changes for the editor policy from the OPS minutes into the ticket. [Reportage: Editor policy]

  • OPS were contacted by some people, including from Rapid editor (Meta), about the editor policy.
  • Mikel Maron offered to help.
  • Document has to be moved from Gdoc, to be easily modified.

Suggestions

  • Add in the editor policy the requirement that any new editors must join the software dispute resolution panel. This would apply both to editors which are part of he website code and external editors.
  • The board to decide if the want to make it a requirement of new editors joining SDRP.
  • Add the requirement of new editors joining SDRP, in the "community feedback" section of the policy.

On the policy

  • There has been a discussion about adding mobile editors.
  • OPS are concerned with the technical requirements.
  • OPS can suggest new editors joining SDRP. Making it a requirement has to be a board decision.
  • The policy can be revised later.

On SDRP

  • The software dispute resolution panel was designed to arbitrate on disputes, and was never used.
  • Establishing SDRP and iD joining was a board decision.

Other points mentioned

  • GEO:: is complicated because it's also used by map viewers, as well as editors, and it doesn't allow you to pass anything other than coordinates.
  • JOSM is probably grandfathered-in.
  • Rapid is a distant iD fork.

Action items

  • Tom to amend the draft editor policy document and move it from Gdoc to hack.md
  • OPS to add the SDRP requirement to the Editor Policy draft and see what feedback we receive.

Fastly future projects

Attribution block error tile image

  • Possible. Requires a lot of testing and careful header handling.
  • We could get away with having our error response be an image, because some clients e.g. will display the image despite it being a 403.
    • That was a problem with Leaflet.

Suggestion: Get the test infrastructure working and do iterative tests.

Automatic config backups?

  • We get some special flags turned-on on our accounts, which requires someone in Fastly to change something on our accounts.
  • Fastly have raised the amount of backends we can have - the default is 5.
  • We have main and a backup distribution and changes were made to our second distribution.

Manual back-ups

  • There is a function in Fastly to do exports.
  • Can export global config: it combines all your templates and data objects into a VCL. Once you have a saved copy, you can create a service using it.

VCL:

  • current version 329.
  • 1432 lines

Action item

Grant to download a back-up of Fastly's VCL, every month.

Containerisation of Ridley services, future of containerisation

More time needed for discussion.

Any other business

Brief discussion on nginx + passenger as replacement apache + mod_evasive

  • Tom has some test work
  • brotli compression is unsupported officially in the free version - only supported in the commercial one.

Tom has been making multiple changes to the website. Andy merged last week ~ three or four months of work.

Suggestion: Potentially get rid of mod-evasive, as Nginx has much better rate limiting support.

osm2pgsql expiry (Huge Area issue)

  • Tom's patch upstream to limit expiry tiles was not accepted.
  • Jochen Topf is looking at out-of-memory issue.

Suggestion: we should move to flex backend of osm2pgsql

Ubuntu 24.04 PR by Grant

  • Grant suggested if Tom wants to take over the PR.
  • Some bits missing.
  • Tom has a spreadsheet of things to build.

Grant to park the PR.

Ironbellly

  • Grant would like to move it to Debian.
  • Need to set up a separate gateway.
  • Tom and Grant will work on it tomorrow.

Minting our own Debian packages

Guillaume offered to talk to Florian.

Vandalism

Minutes redacted after request.

Action items reviewed at the beginning of the meeting

  • 2024-05-30 Grant to revert the request timeouts. [Topic: Discussion of false positives hitting DDOS protections due to request timeouts hitting max allowed and DDOS protection.]
  • 2024-05-30 Tom to amend the draft editor policy document and move it from Gdoc to hack.md [Topic: Editor Policy]
  • 2024-05-30 OPS to add the SDRP requirement to the Editor Policy draft and see what feedback we receive. [Topic: Editor Policy]
  • 2024-05-30 Grant to download a back-up of Fastly's VCL, every month. [Topic: Fastly future projects]
  • 2024-05-02 Paul to draft a policy regarding deploying new WordPress sites. [Topic: 20th birthday WordPress site] [Ticket created](https://github.com/openstreetmap/operations/issues/1071)
  • 2024-05-02 Grant to open a ticket about the PR to turn off DNSSEC [Topic: PR to turn off DNSSEC]
  • 2024-05-02 OPS to revisit the OpenMapTiles application.
  • 2024-04-18 Grant to reply to Equinix, restating the issue in a brief form to them. [Topic: Equinix]
  • 2024-02-08 OWG to review the Editor policy during one of the next calls and possibly vote on it. [Editor Policy adding to OpenStreetMap.org]
  • 2023-11-30 Grant to revisit the "policy for purchasing" document, which currently is focused on specs, and add information such as the process for obtaining approval for purchases. [Reportage] Added info: Who Approves / Steps etc -> Grant to create GitHub ticket
  • 2023-11-30 OPS to review the issue of spam reports to ISPs in 6 months (May 2024) -> Grant to create GitHub ticket
  • 2023-05-18 Paul to start an open document listing goals for longer-term planning. [Topic: Longer-term planning]

Action items that have been stricken-through are either completed, or have been moved to GitHub tickets.

Retrieved from "https://osmfoundation.org/w/index.php?title=Operations/Minutes/2024-05-30&oldid=12718"