Operations/Minutes/2023-03-09

From OpenStreetMap Foundation

OpenStreetMap Foundation, Operations Meeting - Draft minutes

These minutes do not go through a formal acceptance process.
This is not strictly an Operations Working Group (OWG) meeting.

Thursday 9 March 2023, 19:00 London time
Location: Video room at https://osmvideo.cloud68.co

Participants

Minutes by Dorothea Kazazi.

Absent

New action items from this meeting

  • Paul Norman to reply to KDE that we're ok with their current tile usage but this might change in the future. [Topic: KDE usage]
  • Paul Norman to check the IPs associated with scraping OSM tiles using a MapProxy user-agent. Will block, if it is one IP and possible. [Topic: KDE usage]
  • Guillaume Rischard to try to provide options for 1G connectivity for AM6 by the next OPS meeting [Topic: Network Upgrades]
  • Paul Norman to draft a deprecation notice and encourage people to move to OAuth2.0 [Topic: OAuth 1.0a support]

Reportage

OSQA containerisation

OSQA is written in an old Django version.

Related to Discourse (which powers community.osm.org)

  • There is a plugin for Discourse (used in community.osm.org) to do OSQA functionality. It is now officially supported and "under active development".
  • The plugin for the OSQA migration is not yet ready, and it hasn't been ready for more than 6 months.
  • There is ongoing discussion with the forum governance team about e.g. who will write the necessary software for a potential import once the plugin for Discourse is finished.

On suggestion to drop action item and drop OSQA

  • The content on help.osm.org will be available in read-only mode so containerisation is still needed.
  • Containerisation is a temporary step to ease maintenance - does not make it more safer or easier to maintain.

Other points mentioned during discussion

  • Ongoing discussion about replacement of OSQA.
  • No decision/plan yet to migrate existing questions on help.osm.org

Decision: continue with OSQA containerisation.

Travel policy

  • The board has proposed a travel policy and asked for thoughts and comments.
  • Text seems to assume that all OWG travel will be for hardware maintenance.
  • Employees could participate in trainings, conferences (e.g. SotM), meet ISP people.

Suggestions

Other point mentioned

  • Grant will travel to Slough to move some servers.

Live editing of the document.

KDE tile usage

KDE is looking for a more official agreement about use of tiles, similar to QGIS.

  • Email from KDE to Operations Working Group in late February 2023.
  • KDE uses our tiles and they would like something more formal, similar to what we have with QGIS, where Nominatim would have been blocked by default, if we didn't have a special arrangement.
  • KDE uses a special alias.

Statistics of tile usage by KDE

  • averaging 75 tiles/sec from the Marble user agent - as of 2023-03-08.
  • averaging 10-15 tiles/sec - additional usage under other user agents.
  • Low cache ratio: 32 tiles/sec.

For comparison:

  • QGIS: 1100 tiles/sec
  • osm.org: 1600 tiles/sec

Other points mentioned during discussion

  • KDE are in the Top5.
  • 1 order of magnitude less than top1 and top2.
  • We're not struggling because of them.
  • OPS had discussed Marble tile usage in the past when they started using our tiles, as many people were playing with it.
  • KDE usage is OK with us, as long as they attribute.
  • They're the people we want to contribute to OSM.

Action item: Paul to reply to KDE that we're ok with their current tile usage, but this might change in the future.

On blocking heavy tile users

Suggestions

  • Paul to communicate more with the OPS team, if he blocks anything.
  • Block the generic user-agent MapProxy https://mapproxy.org/

On suggestion to block the generic user-agent MapProxy

  • MapProxy documentation is lacking for many of the newly introduced features.
  • Someone is scraping with MapProxy: cache ratio 20%.

Suggestions

  • Custom error message for the MapProxy user-agent.
    • Preference to avoid that, as it is a burden.
  • Check if it is just one IP - might block it, if possible.

Other points mentioned during discussion

  • OSM wiki has a MapProxy example on proxying OSM tiles, which needs to be updated.

Action item: Paul to check the IPs associated with scraping OSM tiles using a MapProxy user-agent. Will block, if it is one IP and possible.

Link shared: https://github.com/mapproxy/mapproxy/issues/494

Network Upgrades

No update.

Guillaume and Grant could look at costs and talk to Clement (French ISP) about 1G everywhere.

Switching AMS — Timeline for next steps

  • The status of this topic has stayed unchanged for a long time.
    • This is because we focused on the migration of the content from forum.osm.org to Discourse (on community.osm.org)
  • We could have HE.net very soon if we wanted to.
  • We don't need IP subnet failover/BGB failover.

Suggestions

  • Have one subnet between the Amsterdam and Dublin data centers, for fast fail-overs, without having to do DNS changes.
    • Failing over the DNS is the least of our problems if we have to do a fast fail-over.
    • Complicated and is a lot of work.
  • Define scope of what Guillaume will be presenting: 1G connectivity for AM6.

Action item: Guillaume to try to provide options for 1G connectivity for AM6 by the next OPS meeting.

OAuth 1.0a support

https://github.com/zerebubuth/openstreetmap-cgimap/issues/286

  • mmd wants to deprecate OAuth 1.0a from cgi-map within the next year due to technical debt.
  • OWG might want to deprecate OAuth 1.0a as well, over an extended period of time.
  • Security-wise, we haven't got rid of http basic authorisation.
  • The code we use to implement OAuth 1.0 is unmaintained.

Things to consider

  • Impact: 30 editors used last year by distinct users.
  • Whether we can remove OAuth 1.0a support without bumping versions.

Suggestions regarding timing

  • Deprecate OAuth 1.0a at the next API version.
  • Wait until JOSM has OAuth 2.0 in stable release.
  • Have a long target frame for the deprecation.

Basic authorisation

  • introduced June 2009.
  • was used by forum.osm.org, now gone.
  • used by OSQA.
  • probably used by OSMtools.
  • JOSM can use it - but doesn't by default.
  • Most apps can use it.

Suggested plan

  • Put a general announcement that we're interested in deprecating OAuth 1.0a. Encourage people to move to OAuth2.0
  • Check metrics via log analysis — look at what editors are used and if they support OAuth 2.0
  • Evaluate the scope.
  • Create a timeline.

Action item: Paul to draft a deprecation notice and encourage people to move to OAuth2.0 (Post-meeting addition: OSM-dev - Future deprecation of HTTP Basic Auth and OAuth 1.0a).

Any other business

Cases of programs abusing account creation

There are cases of sites and programs abusing forms intended for people, e.g. account creation form.

Suggestions

  • have something in policy about not automatically submitting forms intended for users.
  • add proper functionality via API, if needed.
  • do not give the impression to people that will contribute code pull requests to the OSM website that they will be auto-accepted.

Other points mentioned during discussion

  • We already support people logging-in to OSM with other accounts (OpenID, Google, Facebook, Windows Live, GitHub, Wikipedia)
  • We need to know the OSM users' email addresses.iron
  • We force people through the OSM website because they need to agree to the contributor terms.
  • They're still using our API to fetch the data.

Hosting South African aerial imagery

Grant has:

  • 12 TB of South Africa aerial imagery (source imagery) and
  • 12 TB of South Africa aerial imagery (processed imagery).

Suggestions for hosting

  • OpenAerialMap
    • was approached and they replied negatively about hosting.
  • have the raw files on S3
    • costly. Storage cost for Standard: 2.3 cents/GB = USD 2500/year for the 12 TB.
  • have the files sponsored as open data.
    • A South African research institute (who are collecting the imagery) offered to host the processed files.
  • use one of the fairly big servers which we have deprecated.
    • Ironbelly: Old and will have to be replaced.
    • Angor: does not have enough space.

Imagery will be uploaded to Faffy but won't be hosted there.

Things to consider

  • Giving preference to South African aerial imagery, over other regions.
  • Cheaper to host elsewhere, if we take into account redundancy and servers cost.

Imagery hosted so far

  • Luxembourg imagery hosted on the Dev server.
  • Some UK imagery (Hampshire, Surrey)
  • Australia reference imagery.

Action items

  • 2023-02-23 [Bureaucracy] Grant to migrate OWG action items to GitHub tickets (https://github.com/openstreetmap/operations/issues/823)
  • 2023-01-12 Grant Slater to experiment with Netbox. [Topic: Asset management]
  • 2023-01-12 Grant Slater to email Slough to see if he can make any arrangements. [Reportage: Decommissioning]
  • 2022-12-15 Guillaume Rischard to make sure Grant does the forum to Discourse migration. [Topic: Forum to Discourse https://github.com/openstreetmap/operations/issues/604 ] # 2022-12-29 In progress. # 2023-01-12 In progress
  • 2022-12-15 Grant Slater to produce a pull request and finish the OSQA one. [Topic: Containerisation of small services https://github.com/openstreetmap/operations/issues/807] # 2022-12-29 In progress.
  • 2022-11-03 [Network upgrades AM6] Guillaume Rischard to talk with Clement / Open Source ISP about reliability and get insurances that the virtualised Layer2 links to Paris will be reliable. # 2022-11-17 Pending # 2022-12-29 Changed from "OPS" to "Guillaume".
  • 2022-09-22 [Network upgrades @ AM6] Grant Slater and Guillaume Rischard to talk to Clement about the options for network upgrades. # 2022-10-06 Guillaume talked with Clement.
  • 2022-09-22 [Network upgrades @ AM6] Guillaume Rischard to provide a schema at next meeting.
  • 2022-09-22 [Network upgrades @ AM6] Grant Slater and Guillaume Rischard to work out all the costs involved for option 1.
  • 2022-09-08 Grant Slater to document Chef testing. [Topic: How to get more people involved] # 2022-09-22 Chef kitchen tests running locally. # 2022-11-03 pending. #2022-11-17 Pending # 2022-12-29 In progress. # 2023-01-12 In progress - one of the rests fails randomly.
  • 2022-07-14 Guillaume Rischard to ask Brian Sperlongano about OpenMapTiles and YAML, to do a test run. [Topic: Vector tile status]. # 2022-09-22: Suspended for a while. Brian is back at school and has no time to work on this. Guillaume will try to move vector tiles forward with someone else. # 2022-12-29 Still suspended.
  • 2020-12-02 [AWS] Grant Slater to develop some thoughts on what is next for us using AWS. [Topic: AWS] # 2021-05-19 & 2021-06-02 & 2021-06-16 postponed for a few weeks. # 2022-12-29 In progress.
  • 2020-07-29 [AWS] Grant Slater to enable background sync to AWS S3. [Topic: Ironbelly] #2020-08-12&26 & 2021-06-02 Manually run, automated scripting to be added. # 2021-05-19 Grant to run the script again. # 2022-04-09 Still manually run. # 2022-09-22 wants to make sure there are absolutely minimum permissions. # 2022-12-29 In progress. # 2023-01-12 Still manually run. Grant to look at the policies together with Paul.
  • 2020-07-01 Paul Norman to create a ticket about solutions to reduce incoming communications. [Topic:Revision of acceptable use policy to reduce incoming comms] # 2021-05-19 decision to leave the action item open. # 2021-06-02 discussion about priority for account deletion. # 2022-04-09 Grant can show Paul how to do that with autoresponder which Tom built. Might be better to work on an online form (action item below).
  • 2020-07-01 Grant Slater to work out some of the questions for an online form as a solution to reduce incoming comms. [Topic: Revision of acceptable use policy to reduce incoming comms] 2020-08-12 need to think about the reply # 2021-05-19 decision to leave the action item open. # 2022-04-09 Grant is thinking about examples. Suggestion to add what is considered large for tile usage.
  • 2020-04-10 Grant Slater to work out a table of different data bits, work out how they are backed up and what can be potentially improved. [Topic: High Availability / redundancy of OpenStreetMap.org (and primary services)] # 2021-05-19 decision to leave the action item open. # 2021-06-02 pending.

Meeting adjourned 58' after start.

Next meeting

Thursday 22 March 2023, 19:00 London time, unless rescheduled.

Operations meetings are currently being held every two Thursdays, at 19:00 London time.
Online calendar showing the OPS meetings.

Retrieved from "https://osmfoundation.org/w/index.php?title=Operations/Minutes/2023-03-09&oldid=10521"