Operations/Minutes/2024-06-27

From OpenStreetMap Foundation

OpenStreetMap Foundation, Operations Meeting - Draft minutes

These minutes do not go through a formal acceptance process.
This is not strictly an Operations Working Group (OWG) meeting.

Thursday 27 June 2024, 19:00 London time
Location: Video room at https://osmvideo.cloud68.co

Participants

Minutes by Dorothea Kazazi.

Not present

New action items from this meeting

  • Grant to send an announcement about the OAuth status on Monday. [Topic: OAuth]
  • Paul to see if we're still appropriately balanced on the CDN and then OPS to decide on upgrading the RAM of Odin. [Topic: rhaegel usage?]
  • OPS need to do capacity planning for tile.openstreetmap.org [Topic: rhaegel usage?]

Matomo - Microsoft access request

Branko Kokanovic (Microsoft) talked with Paul at SotM US 2024 and asked to do some stuff with Matomo. Microsoft started AB testing the single-sing on (SSO) and the results are suboptimal. Only 35% of people who encounter the OSM sign-in proceed to complete the sign-up process.The reason for this drop-off is unclear to MS, prompting a request for analytics from the OSMF side. They wish to use Matomo to see the number of visitors and sign-ups, to better understand the sign-up funnel. Also the MS team offered to do other analytics that the OPS lack the time to perform. Currently, OPS do not use Matomo.

Options include

  • Provide answers.
  • Grant them access to Matomo.

Concerns

  • Setting a precedent, as access is not typically granted to others.
  • Privacy/personal data issues.
  • Corporate nature of the request.

Information logged by Matomo

  • Account creation statistics are logged.
  • Matomo does not display the full IP address, discarding the last two bytes for IPv4 addresses.
  • Geolocation is performed before discarding the last two bytes of IPv4 addresses.
  • Log-ins are shown in real-time.
  • If a user is logged in, the UserID is anonymised in Matomo, showing only a hashed version.
  • We do not log user progress through the sign-up process, as this feature is unavailable in the free version of Matomo.

Other Matomo info

  • Matomo shows that the copyright page is the top landing and exit page, with a bounce rate close to 100%.
  • Fewer than 1% of site visitors sign up.
  • With the removal of the terms page, the sign-up funnel is minimal. Users must fill in the form and confirm their account.

Suggestions

  • Require non-disclosure agreements (NDAs) to be signed.
    • An NDA alone is insufficient. Changes would have to be made to the Privacy Policy https://osmfoundation.org/wiki/Privacy_Policy if Matomo access extends beyond internal use.
    • It might boil down to trust, as OSMF cannot audit Microsofts's use of the data once access is granted and will not know who within Microsoft is accessing it.
      • Access would be granted to individuals, not to Microsoft as an entity.
  • Get input from the LWG.
  • Have Microsoft lawyers review our privacy policy and provide an opinion on the appropriateness of granting access under an NDA.
    • This review could be done, but it might take several months.

Decision seemed to be

  • Microsoft to ask LWG whether our privacy policy allows granting Matomo access to Microsoft
  • Branko offered to initiate the legal review process within Microsoft in parallel to assess the implications of OSMF granting access to Matomo under an NDA.

SotM EU 2024 attendance

Paul is attending SotM EU 2024 in Poland. Tom and Grant are currently undecided.

OpenMapTiles

Postpone to future call. We require additional time to review the detail.

They replied.

  • Opposed to adding anything other than OSM Carto clone.
  • There are no answers on the status of their licence change or the branding issue.

On uniqueness

  • The cartography is intended to be identical to an existing layer.
  • Technically it is unique, as vector tiles are inherently different from raster tiles.

Other points mentioned during discussion

  • New vector style might be introduced.

Decision: Look at their answer in more detail.

OAuth Status

Brownouts continue. Editors are now generally good-to-go.
1st July go-date seems ok. Grant to push out an announcement.

  • Tom ran it overnight from midnight to 07:00 UTC.
  • Potlatch 3 ok now.

Action item

Grant to send an announcement about the OAuth status on Monday.

rhaegal usage?

Server Located at Description System Distro Stats
rhaegal Hosted by CARNet in Zagreb, Croatia Supermicro X9DRW Debian 12 prometheus

rhaegal as tile render server or keep for future vector?
Also discussed: https://GitHub.com/openstreetmap/operations/issues/1105

  • We had reserved rhaegal for Vector Tiles, but Paul happy with Faffy (dev server).
  • Preference to not have rhaegal as a raster tile server.

On the suggestion to add more rendering capacity in Europe

  • rhaegal is less powerful than Odin and Ysera, which have queues. When they haven't got a mass dirty event, they're not dropping anything, just using the queues as a buffer.
  • The increase in load is being driven by the fact that we're using more accurate expiry now. The increase was when we switched to pgsql expiry.
  • Look at performance of Ysera vs Odin.

On Odin and Ysera

Server Located at Description System Distro Stats
odin Equinix Amsterdam Tile server Supermicro SYS-1029P-WTRT Ubuntu 22.04 prometheus
Ysera University College London Tile server Supermicro SYS-1029P-WTRT Ubuntu 22.04 prometheus
  • Odin and Ysera are past their predicted life-span. We had specced them out to last 5 years.
  • Number of coses: 28 cores.
  • CPUs: 6-7 years old.

Suggestions:

  • Upgrade Odin and Ysera to the same clases as newer European render servers.
  • OPS to look at performance of Ysera vs Odin, as Ysera now has more RAM.
    • Some improvement ~ 10% extra performance.
    • There's an open ticket. Cost: EUR 260+ to upgrade the ram + remote hands cost.
  • Could double the number of calls in the system and go slightly higher clock rate, but it's probably about a GBP 1000 per machine.
    • Not worth it.

Other points mentioned during discussion

  • At worst we're serving about 5% dirty tiles of what hits the back end, and missing virtually none.
  • Rams equally clocked at both machines.

Discussion on graphs
- Missed tiles: Where we served a 404 because we were not able to render it in time.
- Dropped tiles: where the tile was dirty, but the queue was really full, so we have failed to queue it.

Action items

  • Paul to see if we're still appropriately balanced on the CDN and then OPS to decide on upgrading the RAM of Odin.
  • OPS need to do capacity planning for tile.openstreetmap.org

"OSM PPA" work approval

Grant ok to build "OSM PPA" for Debian 12 using GitHub actions?
Will use GitHub actions private runner for ARM64 builds until 2025 (Catford hosted cluster of Raspberry Pi 5)
Estimate of work required 1 week.
Based on https://salsa.debian.org/salsa-ci-team/pipeline (builder) (debootstrap)

Florian https://wiki.openstreetmap.org/wiki/User:Flohoff had sent an email.

Suggestions

  • Discuss the design offline.
  • Use package.io - is a commercial repository, but they do have an open source option.

On arm building

  • Discuss the design
  • Currently it is a tricky requirement, as if you use emulation with GitHub actions, it is extremely slow (GDAL takes 6 hours to build and hits a timeout). It will probably be similar for Mapnik.
  • Grant has some scripts for private testing.

Other points mentioned during discussion

  • Build step is some wrapper shell scripts.
  • The DSC tells it where the source is and it downloads the source from there and builds it.
  • Estimated amount of work: 1 week.
  • 5 packages to be built. Some are just available in backports. CGImap is blocking us moving with the six front ends.

Feedback

Tom and Paul in favour of Grant working on OSM PPA, as it will allow us to unblock a lot of Debian moves.

Action item

Grant to check whether the free version of package.io will cover us.

Editor policy public current draft?

No push back from the Software dispute resolution panel (SDRP).

Suggestions

  • Publish draft for review.

Other points mentioned during discussion

  • Opinion expressed opposing the foundation putting a requirement that someone has to join the SRDP.
  • Related comment by R. Fairhurst email pasted in the chat during the meeting.

Points to consider

  • whether requiring to be a member, would be a conflict with the SDRP's mandate and,
  • whether it would also be at odds with the foundation's mission to support but not control the project.

Action item

Grant to publish the updated draft for feedback and we will review at next Ops call. Feedback period 2 weeks.

Fastly

Fastly wants us to use things like their image tool processing tool "image optimizer" to serve webP when the browser supports it. Fastly's aim: For communication purposes.

On Fastly's "image optimizer"

  • Can change the format of your image based on whether or not the client supports webP.
  • Is not enabled on our account yet, but can be enabled in a couple of minutes.

On image file formats

  • If we're going to serve webP and PNG, we should be generating webP and then converting it to PNG.
  • webP can be faster to generate with mod-tile in some instances than a compressed PNG, depending on the compression.

Other points mentioned

  • Most browsers can handle it well if they are instructed to load a PNG file but are actually served a WebP file.
  • Most browsers since IE5 do content sniffing.

Concern: Would need to be open source, to power www.osm.org. Compute and the image processing stuff are different from the CDN.

On visualisation example mentioned

  • Example of what could be created: map showing a percentage of our tiles, as flashing squares. Or a heat map.
  • Tom had produced a nice map showing tile usage in Prometheus or Grafana. It was based on the metrics that Fastly provide of how many tiles they were serving from each cache.
  • The visualisation probably does not have to be in real time, but might be nice to have.

Issues:

  • We do not know exactly what they want.
  • We probably do not have the bandwidth to take such a project on.

Fastly:

  • They might be more interested in where the people were that were requesting tiles rather than what tiles they were requesting.
  • Could do the sampling and send a percentage of logs.
  • Do not want access to our logs.
  • They have some visualisation people.

Other points mentioned

  • Map on Mastodon showing the date shifting, posted by Paul. However, it is not suitable for real time.

Contract

  • 9k/month pretend value, excluding the actual bandwidth.

Action items reviewed at the beginning of the meeting

  • 2024-06-13 Grant to email SDRP and copy OPS and Guillaume
  • 2024-05-30 OPS to add the Software dispute resolution panel requirement to the Editor Policy draft and see what feedback we receive. [Topic: Editor Policy]
  • 2024-05-02 OPS to revisit the OpenMapTiles application.
  • 2023-05-18 Paul to start an open document listing goals for longer-term planning. [Topic: Longer-term planning]

Action items that have been stricken-through are either completed, or have been moved to GitHub tickets.

Retrieved from "https://osmfoundation.org/w/index.php?title=Operations/Minutes/2024-06-27&oldid=13002"