Operations/Minutes/2020-08-12

From OpenStreetMap Foundation

Draft notes

OpenStreetMap Foundation, Operations Meeting* - Agenda & Minutes
Wednesday August 12th 2020, 18:00 London time
Location: Video room at https://osmvideo.cloud68.co

* Please note that this was not strictly an OWG meeting.

Participants

Present:

Apologies:

Minutes by Dorothea.

Administrative

Previous minutes

2020-07-29

Action items from this meeting

  • Tom to document correct process of updating planet diff files and steps on operations website. Suggestion to move state file last.
  • Paul to draft text on things done to improve situation with Ironbelly and share it with the OWG.
  • Grant to sent announcement for brief outage: Sunday 16th, 11:00 am UK time. Message for one hour window.
  • Michal to try to rekindle excitement about people helping with imagery (on dev channel/imagery channel or Slack).

Reportage and action item updates

Fastly

Ian's update: waiting for a new contract from Fastly. I've asked them for more credits to run a test before we need to sign anything.

Forum migration

Is a bit stalled. Hopefully Ian will have more time this weekend to work on it.

Costs/bandwidth for S3 serving planet files

  • ~ 3,000 USD/month. Fairly expensive due to Amazon bandwidth costs. Storage cost not included - minimal in comparison. Without mirroring: ~ 15,000 USD/month.
  • Ironbelly replacement: 30-40,000 USD for a pair of servers for redundancy.
  • 33 TB/month of bandwidth

Possible ways forward

  • Hybrid approach: Serve the most recent ones out of cheaper bandwidth (they do not necessarily constitute most of the requested files) and an object store for older ones?
    • Advantages: Single synchronised storage, duplicated. Older files are immutable(ish).
    • We should work out how to calculate AWS cost? How much new data downloaded daily?
  • Is there someone who offers cheaper storage solution?
  • Cache Storage vs AWS Bandwidth...

Allan joined ~12' after start

Bittorent planet file

  • Christian busy. Has provided examples of commands.
  • Guillaume has pinged Christian.
  • https://github.com/openstreetmap/operations/issues/451
  • Careful if we select public tracker.
  • If we run tracker, we get the metrics.
  • Amazon S3 tracker: no longer supported, max filesize 5 GB.

hCAPTCHA

  • hCAPTCHA people reached out and happy to help if needed. They have commercial paid features, may be willing to give us if asked specifically.
  • Currently blocked on this: waiting for Mediawiki 1.35 to come out.

Quote up a server to replace errol/kessie

Errol (UCL)

  • A new person in OWG asked to do Errol.
  • Need to be replaced at some point.

Kessie

  • Move imagery storage to an object store?

Reach to AWS

  • Michal feels blocked, could draft something.
  • We got contacted by AWS account manager. Haven't talked to them yet.

AWS has mirror of planet on S3.

  • Athena: planet and some tables, kept up to date. You can query the tables - doesn't have geometries.
  • Probably no new features. Seth Fitzsimmons used to work on it but now works in different part of AWS.
  • S3 bucket that mirrors the planet is available to AWS subscribers in a way that's faster for them to download where they pay for accessing the data. We could ask them.
  • We don't have anything unique to offer them as selling point.
  • Maybe they want to retire the public S3 bucket or let us manage it for them.

Issue with planet diff files

Action items:

  • Tom to document correct process of updating planet diff files and steps on operations website. Suggestion to move state file last.
  • Paul to draft text on things done to improve situation with Ironbelly and share it with the OWG.

Brief outage announcement

  • Sunday 16th, 11:00 am UK time.
  • 5' outage of some services. If something goes wrong, period will be extended.
  • Stop diff generation service, create rsync, mark gpx as read-only.

Action item: Grant to sent announcement for brief outage: Sunday 16th, 11:00 am UK time. Message for one hour window.

Imagery - getting others involved

  • Michal worked with Grant to put together an announcement.
  • Initial strong excitement. Need to coordinate.
  • Ideal scenario: open aerial layer on osm.org
  • People can give us imagery but they cannot host it for us.
  • Lamda: proprietary system.

Suggestion: Find dataset to do trial. New Zealand's mapping agency has huge amounts of aerial imagery and are interested in COG format. Guillaume has some LIDAR stuff planned that could be produced.

Action item: Michal to try to rekindle excitement about it (on dev channel/imagery channel or Slack).

Open Ops Tickets

Review open, what needs policy and what needs someone to help with... https://github.com/openstreetmap/operations/issues

Diff replication

https://github.com/openstreetmap/operations/issues/154
Jochen is working on it.

List of tier 1, 2, 3 core OWG services

https://operations.osmfoundation.org/policies/classification/

Suggestion: OWG to refresh their list of tier 1,2,3 services (does not include editors)

Next meeting

Wednesday, 26th of August 2020, 18:00 London time

Retrieved from "https://osmfoundation.org/w/index.php?title=Operations/Minutes/2020-08-12&oldid=7604"