Operations/Minutes/2023-11-30

From OpenStreetMap Foundation

OpenStreetMap Foundation, Operations Meeting - Draft minutes

These minutes do not go through a formal acceptance process.
This is not strictly an Operations Working Group (OWG) meeting.

Thursday 30 November 2023, 19:00 London time
Location: Video room at https://osmvideo.cloud68.co

Participants

New action items from this meeting

  • Grant Slater to revisit the "policy for purchasing" document, which currently is focused on specs, and add information such as the process for obtaining approval for purchases. [Reportage]
  • Tom Hughes to check whether OSMers with Facebook logins got null passwords or random strings that don't function as passwords. [Topic: Facebook logins]
  • Paul Norman to open an OPS ticket on the logins which have passwords stored as md5 or salted md5. [Topic: Logins which have passwords stored as md5 or salted md5]
  • OPS to find out how many users have null password or old passwords. [Topic: Logins which have passwords stored as md5 or salted md5]
  • OPS to review the issue of spam reports to ISPs in 6 months (May 2024) [Topic: Spam reports to ISPs]
  • Paul Norman to post on the PR https://github.com/openstreetmap/openstreetmap-website/pull/4379 and do a revision of the operations' policy. [Topic: Hosting provider credit policy]

Reportage

IPv6 - Getting a /48

Action item: 2023-11-16 Grant to ask if we can get one /48 and get routing to both DUB and AMS. This action item was added because there's a security company in the same /48 as us that were doing white hat scans and got reported. This resulted in the whole /48 getting blocked, including us.

  • We wanted to do get our own /48 for spam protection.
    • This would be costly.
  • Grant did not proceed with this.

Decision: Will not pursue further.

Specifications list when machines are ordered

Action item: 2023-11-16 Grant to start a list when machines are ordered. [Topic: Raid controller cards]

  • We could have gotten better controllers for the Oregon State University machines.
  • The list is expected to change each time an order is placed.
  • It is advisable to overspec in order to account for future needs.

Suggestion: Do not use Raid controllers.

Action item: Grant Slater to revisit the "policy for purchasing" document, which currently is focused on specs, and add information such as the process for obtaining approval for purchases.

OPNVKarte featured layer

Related to https://github.com/openstreetmap/operations/issues/907

Pull request not updated yet.

Open document listing goals for longer-term planning

Action item: 2023-05-18 Paul to start an open document listing goals for longer-term planning. [Topic: Longer-term planning]

Will be done next year.

Facebook logins

  • Facebook will terminate our access on January 8, 2024, unless OSMF registers as a company on their system.
  • Multiple attempts have been made to register OSMF. Despite the application process indicating a verification timeframe of a couple of days upon document upload, applications are consistently rejected three weeks later.
  • The issue has been escalated internally at Facebook by Angelina Calderon, Facebook's representative on the OSMF Advisory Board.
  • Guillaume will provide additional updates and will initiate contact with Facebook if there is no response.

Suggestions

  • Obtain an internal ticket reference number, initiate a new request for approval, and reference the new request in the communication.
    • No space is available for additional comments.
  • Set a deadline for deactivating this feature if no response is received.
  • Revisit at the beginning of January.
  • Cease the creation of new accounts with Facebook login and communicate the discontinuation of this feature.

Action item: Tom Hughes to check whether OSMers with Facebook logins got null passwords or random strings that don't function as passwords.

Logins which have passwords stored as md5 or salted md5

See post-meeting issue Invalidate old passwords #1006

  • We are storing unsalted md5 passwords of people who have not changed their password since 2007, or not logged in since 2013.
  • There is no need to flag those accounts, setting their passwords to null is sufficient.
  • Tom was the person who initially identified that passwords were unsalted and added salts.

Suggestions

  • Discard weakly protected passwords. All users with a null password will be asked to perform a password reset during their next log in.
    • This will also apply to users who used a social media login and have not set up a password.
  • The wording of the page when the users try to login can be figured out at a later step.

Action items:

  • Paul Norman to open an OPS ticket on the logins which have passwords stored as md5 or salted md5.
  • OPS to find out how many users have null password or old passwords.

Spam reports to ISPs

Background

  • Previously, spam reports may have been received, but Cogent was filtering them out.
  • Users are not intending to report us to our ISP. It's their email provider doing it on behalf of them.
  • Reported spam messages are frequently relayed by us rather than originating from us.
  • Legitimate emails flagged as spam often end up in the osmfoundation's spam queue, requiring manual intervention by Grant for removal.
  • There was a small spike on 2024-10-27, with 1.5% of e-mails sent to Gmail was flagged to be spam, which was an outlier. Usually it's arounf 0%.

Types of flagged emails

  • Emails originating from community.osm.org (Discourse)
  • Emails originating from www.osm.org, including notifications related to sign-ups.
    • Could also be cases where the DWG is doing a big revert and the person is deliberately marking all the changeset comments as spam.
  • Emails from digest mode, as there's no easy link to unsubscribe someone from it, though it sends them an email to confirm.
    • Tom can sunsubscribe users from digest mode.
  • WeeklyOSM emails get increasingly often marked by Gmail as spam.
    • The small spike on 2024-10-27 did not correlate with a weeklyOSM release.
    • Suggestion: Consider asking them to stop sending those to the mailing lists if they're getting increasingly reported as spam.

Other points made

  • In Gmail and there's a list unsubscribe header, then Gmail will offer to unsubscribe you.
  • A lot of people use the spam button in Gmail when they don't want to receive emails, even if they have subscribed to them.

On emails by community.osm.org (powered by Discourse), flagged by users as spam

  • User flagging behavior:
    • Users are flagging as spam genuine emails sent by the website or by community.osm.org
    • Some instances involve email messages within Discourse threads written in a language different from the thread, leading users to flag them as spam.
  • Bouncing email handling: If emails bounce for a community.osm.org user many times, Discourse stops sending emails.
  • Policy on flagged emails: We're not supposed to keep sending those emails after they've reported them as spam, but that means the users can't use the website or Discourse.
  • Unsubscribe functionality: All e-mails from community.osm.org have an "unsubscribe" function.
  • Current actions: Grant has taken the initiative to unsubscribe users from Discourse emails if they have reported such emails as spam.

On emails from www.osm.org flagged by users as spam

  • The website lacks functionality for users to manage their email preferences, preventing them from opting out of certain email communications.
  • We should ensure that all communications passing through the website messaging system remain visible as messages on the website.
  • We could in the future make it possible to stop sending emails for notes, changset comments and diary entries.
  • Emails will still be send for password resets and sign-up confirmations.

Suggestions

  • Collate reports to identify common patterns among flagged emails.
  • Evaluate whether a limited number of individuals are responsible for flagging emails as spam.
  • Consider unsubscribing users to address the issue.
  • Draft a policy about ceazing to email users with bouncing addresses.
  • Discontinuing emails to users s who haven't interacted with the website for a year.
    • We lack a reliable measure for user interaction.

Comment that most of the above would make the situation more complicated.

Policy consideration

  • Consistent reporting of emails from www.osm.org as spam to our ISP should be considered incompatible with having an OSM account.

Decisions:

  • Continue unsubscribing community.osm.org users from Discourse emails, if they flag them as spam.
  • Review the issue in 6 months.

Hosting provider credit policy

Related to https://github.com/openstreetmap/openstreetmap-website/pull/4379

  • Sarah Hoffmann submitted a pull request (PR) https://github.com/openstreetmap/openstreetmap-website/pull/437 to add OSMF corporate members as as a distinct group in the website credit list.
  • The existing policy about the organisations credited on www.osm.org is intended to reflect the benefits they're providing to the operations, such as donating hardware or hosting.
  • Fastly (currently on top 3)
    • provides services amounting to 100,000 per month.
    • they will not sign up as Corporate Members.
  • Amazon
    • Hosting planet.
  • Potential top3: Fastly Amazon, OSU.

Suggestions

  • Credit the top two contributors first, followed by acknowledgment of Corporate Members.
  • Remove UCL, Fastly, Bytemark from the existing list and have the addition to the www.osm.org credit list as a benefit to Corporate Members.
  • Feature Fastly and the Corporate Members.
    • Securing an agreement from Amazon may pose logistical challenges.

Action item: Paul Norman to post on the PR https://github.com/openstreetmap/openstreetmap-website/pull/4379 and do a revision of the operations' policy.

Microsoft MapBuilder accounts

  • The board's passed a resolution to go ahead with the automatic creation of accounts and we need to notify the board that we don't have the technical means to whitelist.
  • This issue does not fall under the scope of operations of the OWG but of the website maintainers.
  • OPS could advice the website maintainers, but the IP issue is complicated.
  • Tom has replied to the MapBuilder team in the past, explaining the technical limitations.

Suggestion:

  • Notify them that there's no whitelist functionality, but we would accept PR for now.
    • OPS cannot accept the PR. This is up to the website maintainers, Tom and Andy.

Feedback by Tom

  • Could be done using the ACL table that already exists.

Decision: Leave it to the osm.org website maintainers, Tom Hughes and Andy Allan.

Recovery Report

Grant is working on a draft document with:

  • services we have
  • how they are backed up
  • associated risks
  • time estimate for recovery and
  • our recovery plan.

- Grant intends to have this is a public document.
- Concern about publicly listing weaknesses.

Next steps include

  • standardising documents.
  • splitting services that we run and required resources.
  • pushing backups to the cloud.

On OSM website: consider how we do a base backup into our flow

  • A base backup combined with the writer head logs provides extra functionality.
    • It also makes the database restore slower.

On PGbackrest https://pgbackrest.org/
There is an open question on if we should also have pg_basebackups when combined with WAL backups to allow point-in-time recovery. Likely using https://pgbackrest.org/ sending the backups to S3.

  • Additional complications compared to the other systems.

On enabling checksums into our postgres database
PostgreSQL supports integrity checksums in data read and written, likely available from version 14 onward.

  • A complete database reload might be necessary to enable checksums.
    • It might be possible to enable checksums for future data exclusively, excluding past records.

Action items reviewed at the beginning of the meeting

  • 2023-11-16 Grant to ask if we can get one /48 and get routing to both DUB and AMS. [Topic: IPv6 related]
  • 2023-11-16 Grant to start a list when machines are ordered. [Topic: Raid controller cards]
  • 2023-11-02 Paul to update the PR https://github.com/openstreetmap/openstreetmap-website/pull/4126 [Topic: OPNVKarte]
  • 2023-09-21 Grant to create a table on the cache headers that we send. [Topic: Surrogate key patch]
  • 2023-06-29 Grant to put Martijn van Exel's policy for addition of OSM editors to the osm.org menu out for feedback.[Topic: Draft policy by Martijn]
  • 2023-05-18 Paul to start an open document listing goals for longer-term planning. [Topic: Longer-term planning]
  • 2023-05-04 [WordPress] Grant to share list of WordPress users with Dorothea and their response to keeping an account. [Topic: WordPress security] - Shared, but additional work required. Turned into an issue https://github.com/openstreetmap/operations/issues/1005
Retrieved from "https://osmfoundation.org/w/index.php?title=Operations/Minutes/2023-11-30&oldid=11741"