Attendees
| IRC nick
|
Real name
|
| apmon |
Kai Krueger
|
| Firefishy |
Grant Slater
|
| iandees |
Ian Dees
|
| pnorman |
Paul Norman
|
| tmcw |
Tom MacWright
|
| TomH |
Tom Hughes
|
| zere |
Matt Amos
|
Summary
- Notes branch issues
- In a previous meeting it was agreed that closing an issue should require authorisation, but this is the first browser-based method to require it and there is no pre-existing mechanism.
- Potential mechanisms are:
- A one-time token + cookie, although great care would need to be taken to avoid XSS / forced browsing attacks.
- OAuth in the browser: [1] + [2]
- Agreed: use OAuth for auth-requiring methods from the browser, including closing a note.
- 2013 goals
- Agreed: Current hack event subsidies are not sufficient to cover events where donated facilities cannot be found, therefore EWG will make available a subsidy of around $500 to cover food + venue.
- There was discussion of whether travel should/could be covered as well, but there was no sufficient consensus to agree it.
- Other ideas for improvements:
- Automatic planet-latest downloads to errol, so that planet files are available locally to cut down setup times and reduce duplicate downloads.
- Add tracking for the replication lag for the osm2pgsql database to errol's munin, so that people can judge how up-to-date the hosted instance is there.
- Action: pnorman to check if there are any blockers to generating OSMCoastline and look at generating ocean polygons rather than land polygons.
- Time of the next meeting
- The earlier time (since DST changes in North America) is more convenient for some attendees, so a vote was taken on whether to keep this new time.
- Agreed: next meeting 2013-04-01 17:00 UTC
IRC Log
18:00:57 <zere> minutes of the last two meetings: http://www.osmfoundation.org/wiki/Working_Group_Minutes/EWG_2013-02-25 and http://www.osmfoundation.org/wiki/Working_Group_Minutes/EWG_2013-02-18
18:01:14 <zere> as ever, let me know if there's anything needs changing with them.
18:02:39 <zere> this meeting we really have to get a set of goals sorted out - not only because they help others within and without OSMF figure out what we're trying to do, but also so that we can request resources that we might need to reach those goals
18:03:05 <TomH> can we have a quick discussion on the last of the notes blockers first?
18:03:17 <zere> sure, as long as it's quick ;-)
18:03:23 <zere> #topic notes branch issues
18:03:28 <zere> TomH: go for it
18:03:30 <TomH> so we agreed on requiring auth for close
18:03:45 <TomH> but api calls traditionally are authed by http auth or oauth
18:03:58 <TomH> but when closing from the web frontend the user will just have cookie auth
18:04:40 <TomH> so either we allow cookie auth just for that api call, or for all api calls, or we come up with some other solution
18:04:52 <zere> we must have the same problem for things like user prefs / GPX traces, no?
18:05:05 <TomH> no, because the web site doesn't use api methods for those
18:05:28 <zere> you mean the website accesses them directly?
18:05:34 <TomH> there is a GPX api but there is a parallel set of methods for the web site
18:05:44 <pnorman> I know for other API calls and looking at redactions I've wished for cookie auth
18:05:49 <TomH> and user prefs are never accessed from the web site
18:06:06 <TomH> pnorman: yes it drives me mad when I go to an API object in my browser ;-
18:06:52 <pnorman> I set up some stuff to do it easily with http auth and CURL, but for just viewing some v1 redacted node it's more work than it should be
18:06:55 <zere> it sounds like it would be most useful to include cookie auth for api methods, then?
18:07:19 <zere> any downside to doing this?
18:07:36 <pnorman> tmcw: ping, you're working on a significant browser-based editor, any thoughts on cookie auth?
18:07:43 <TomH> XSS I guess
18:08:07 <TomH> third party site doing a write request to the API and piggy backing on an existing cookie
18:08:09 <tmcw> pnorman: never used it w/ osm - oauth only
18:08:57 <tmcw> why can't the web frontend use oauth as well? I haven't looked into potlatch 2 in detail, but seems like that's how it does things?
18:09:24 <pnorman> I was going to suggest cookie auth for read only to avoid XSS, but that doesn't solve the notes problem
18:10:02 <TomH> yeah - think I will have to cook up some magic with a token
18:10:58 <pnorman> tmcw: I don't think P2 on osm.org uses oauth, it never asked me to authorize it
18:11:10 <TomH> it does, but it is pre-authorized
18:11:19 <tmcw> y, which this would be as well
18:11:41 <TomH> the server automatically authorizes it and embeds the key and token in the page taht starts it
18:12:37 <zere> is it possible to get access to all the information & headers needed to compute the oauth signature from within the browser JS environment?
18:12:53 <TomH> well iD does it, so sure
18:13:16 <TomH> but I think tmcw wrote a whole bunch of code to do that ;-)
18:13:39 <tmcw> https://github.com/osmlab/osm-auth + https://github.com/osmlab/ohauth
18:13:49 <zere> i haven't looked at the iD internals, so i wasn't ruling out the possibility it needed a helper server / service to do it.
18:14:02 <TomH> I'll look into it - suggest we move on now
18:14:25 <tmcw> zere: nope, pure js. ohauth is a low-level interface. the tricky part is sha1 and there was already a solid impl around
18:14:50 <TomH> oh I know all about that.....
18:15:50 <Firefishy> Would it be possible to just not display the close button/link if not logged in?
18:16:29 <zere> haha, but not auth? that's uhh... an interesting solution
18:16:38 <tmcw> Firefishy: sure, kind of security via obscurity though
18:17:05 <Firefishy> Cookie auth sounds lovely for XSS.
18:17:47 <zere> indeed, that's why we'd have to embed a token as well. sounds like a pain to me.
18:18:24 <Firefishy> Or close becomes /login?nexturl=close-link ?
18:18:35 <pnorman> I think oauth would be better if we can get it going in the browser, it has the plus side that it makes it easier to use the osm.org implementation elsewhere
18:19:03 <zere> Firefishy: that still has the same XSS properties, unfortunately.
18:19:04 <tmcw> (which could just be ohauth..)
18:19:20 <tmcw> sidenote: if anyone wants access to osmlab/ohauth / the organization, let me know
18:19:37 <tmcw> I would add everyone but github is annoying with notifications so I don't want to spam
18:21:35 <pnorman> I believe you can turn notifications off
18:22:06 <tmcw> not on a admin-level afaik, so people have to do it themselves
18:23:05 <zere> and then if someone creates a fork, you still get auto-subscribed to none or all... there's no finer-grained control at the repo level that i've been able to find
18:23:52 <tmcw> it is a bummer that it's not more fine-grained. but I'm trying to write reusable modules here, so use them plz
18:23:53 <tmcw> :)
18:24:07 <pnorman> anyways, what's the conclusion? tomh to write code using oauth for notes branch?
18:25:15 <tmcw> seems like it
18:26:35 <zere> #agreed use oauth for auth-requiring methods from the browser, including closing a note.
18:26:41 <zere> #topic 2013 goals
18:27:14 <zere> we came up with 3 before: 1) place for technical discussion, 2) better documentation, 3) better availability of issue-tracker information
18:27:35 <zere> i was talking to pnorman and he suggested further that we want to support hack events
18:27:55 <pnorman> Get at least 1-2 more people in the meetings would be another suggestion
18:28:16 <zere> but that the funding available in 2012 (100 GBP per event) was far too low to cover in cases where the venue wasn't donated
18:28:21 <tmcw> there's the chicago hack day coming up in the us
18:29:32 <pnorman> to cover anything more than an order of pizza, I'd guess $500-$1500 per event, depending on what was being funded
18:30:13 <zere> we figured that $400-$500 would probably cover some food and venue, without bein extravagant
18:30:51 <zere> the next question is whether some form of travel subsidy would be useful - would be worthwhile from OSMF's point of view
18:31:06 <pnorman> abstaining from this.
18:31:54 <zere> anyone else have any thoughts about this?
18:32:58 <tmcw> not really - it'll be more actionable when there's a new hack day planned that needs funding
18:33:49 <zere> sure, but if there's just silence here then it sounds like nobody is interested and, when the new hack day is planned, the funding won't be available.
18:34:27 <Firefishy> Pizza + beer + cheap venue ok. Travel money... not so sure.
18:35:14 <zere> the travel subsidy is only problematic insofar as it would be difficult to allocate fairly.
18:35:23 <pnorman> Not taking a stance on if it should be funded, but if you don't come to a decision (and let people know) no one will apply
18:35:53 <zere> indeed
18:38:00 <zere> perhaps we could get a quick vote on hack event funding: none / food+venue / +travel - ?
18:38:25 <zere> personally, i like food+venue+travel, but recognise it's problematic.
18:39:16 <pnorman> One way or another decide on if it's worthwhile. MT/board might decide it's too much for the budget, but I'd assume at this point that there's funding for all 3 and if it has to be cut back later so be it
18:39:31 <pnorman> +1 for at least food+venue, abstain for if it should include +travel
18:40:23 <zere> sure, and this is best-guess - it's possible to change later, it's just a faff, and the money might have been allocated elsewhere.
18:42:54 <zere> i'm sensing great indifference to this issue, so on the basis of a paltry 3 votes, i'm moving on...
18:43:26 <zere> #agreed hack event subsidy around $500 to cover food + venue. no travel... perhaps.
18:43:47 <zere> any other resources we feel like we're going to need?
18:43:47 <pnorman> how do we communicate this so that people know to apply?
18:44:28 <zere> i suggest after the budgets / plans have been through whatever process and approved, we write up something for CWG that says what's available and how to apply
18:44:44 <zere> presumably we can handle applications here in the meetings
18:45:15 <zere> sounds good?
18:45:20 <pnorman> k
18:46:32 <zere> any other resources we feel like we're going to need? - perhaps build / CI servers, other events, bug bounties, ???
18:46:46 <pnorman> we should consider bug bounties
18:47:24 <pnorman> not saying we should have them (there are two sides to that) but we should consider them
18:51:13 <pnorman> would feature bounties be a better word? we're pretty good about not having major bugs
18:52:00 <Firefishy> Servers! :)
18:53:29 <zere> Firefishy: servers for anything in particular, or just on principle? ;-)
18:54:03 <Firefishy> principle.
18:54:33 * Firefishy 's gears are turning on what the next DB server should look like.
18:54:40 <zere> pnorman: i have serious doubts about whether bounties actually improve developer engagement, long-term. and whether the level we might have to offer it at would be the best use of the money.
18:55:15 <zere> for example, if a bounty was $250 - would it be better spent as two bugs / issues, or one hack weekend?
18:56:11 <pnorman> I think hack weekends are more effective, seeing what's come out of the last ones. Toronto had refactoring of maproulette, I think the last UK had the start of the carto rewrite (which I hope we can deploy sooner rather than later)
18:59:03 <zere> the positive of bounties is that they're directed effort - for anyone who is motivated by a monetary reward. the negative is that anyone so motivated now has an incentive to leave / push issues to the point where they get a bounty and drive out competition for the bounty...
18:59:18 <pnorman> on the other hand, I guess there's the two sides to it - hack weekends seem to be better at starting projects than finishing
18:59:27 <zere> indeed
18:59:48 <pnorman> what would be a current potential canidate? the carto port?
19:01:13 <pnorman> i guess the carto port is a bit of a weird one because it's not a dev matter so much as a cartography matter and requires a cartographer more than a coder
19:01:51 <zere> not sure - i think that one is pretty "done", isn't it? it's queued along with a bunch of other server stuff to be done... real soon now? Firefishy - do you remember better than i?
19:03:46 <Firefishy> There are a few minor blocking bugs (coastline support requires files off tile.osm.org), but nothing that is holding things up. Freeing up orm + adding SSD are the real blockers
19:03:56 <pnorman> It might be waiting on tile server upgrades
19:05:04 <Firefishy> I also need to get a semi sane quote for Intel DC S3700 800GB disk.
19:05:32 <pnorman> doesn't look like there are any blockers on https://github.com/gravitystorm/openstreetmap-carto/issues/39, is the concern more one about relying on external providers?
19:06:56 <Firefishy> pnorman: well that bug needs code. I believe gravitystorm prefered the route of a sea polygon rather than water, but I may be mis-remembering.
19:07:11 <Firefishy> *sea rather than land
19:08:13 <pnorman> On a related matter, could we get the latest planet at a central location on errol as part of the weekly routine? It seems like downloading it to test coastline generation is likely to duplicate existing data
19:09:20 <pnorman> Not after an archive of all planet files, just the last week or two weeks
19:09:38 <apmon> zere: Regarding servers, I think dev servers might be part of ewg
19:09:53 <apmon> sorry, I missed the meeting earlier...
19:10:27 <apmon> E.g. Wikimedia has a pretty sweet setup with the toolserver and now wikimedia-labs
19:10:58 <apmon> which helps people develop all sorts of things without having to worry about the server resources to deal with such massive databases
19:11:14 <zere> pnorman: i'm sure we can manage that. should be fairly simple cron job...
19:11:29 <apmon> Main servers, I would guess would come from the owg budget?
19:11:37 <zere> #idea automatic planet-latest downloads to errol
19:11:52 <pnorman> Someone (one of us? cwg?) should do a dev announcement about the osm2pgsql DB on errol, with a couple of examples of potential uses
19:12:17 <zere> apmon: sure - but we need to make a better case than "wikipedia has buttloads of servers, so we should too" ;-)
19:12:50 <iandees> as a datapoint, i've tried to let people in the US know about the OSM US server and haven't had a lot of bites
19:13:03 <apmon> ;-) Well, I think one can make a case for it, if there is agreement that it would be useful
19:13:45 <zere> yeah, i think 99% of people develop on their own machines using extracts. if they develop custom schemas then there's not a lot we can do - it's impractical to support 1001 custom databases for the full planet
19:14:06 <pnorman> are we using errol to capacity? i know that top indicates that the main CPU use is the bing imagery analyzer, but I don't think the server is at capacity
19:14:08 <zere> apmon: how's the osm2pgsql database doing - staying up to date?
19:14:34 <apmon> imho having to deal with large resources to develop and test stuff is still one of the bigger issues to developing /implementing tools
19:14:41 <zere> from past experience, i know it's definitely possible to max the i/o out pretty easily ;-)
19:14:49 <pnorman> idea: add tracking for the replication lag for the osm2pgsql database to errol's munin
19:15:15 <apmon> just checking on the osm2pgsql db...
19:15:17 <zere> i think meetbot needs the '#' at the front to pick those up
19:15:26 <pnorman> #idea add tracking for the replication lag for the osm2pgsql database to errol's munin
19:15:34 <pnorman> I might not have the access rights for that
19:16:02 <zere> everyone should... it's supposed to track the channel rather than any one particular person
19:16:05 <apmon> It is about two hours behind, but given that erol seems pretty busy at the moment that isn't too bad
19:16:26 <pnorman> errol IO is maxed right now and disk usage is steadily increasing
19:16:36 <zere> hmm
19:16:37 <zere> ok
19:16:56 <apmon> more SSDs for errol.... ;-)
19:16:57 * Firefishy is to blame for the maxed out IO at the moment.
19:17:30 <apmon> well, running a plannet wide osm2pgsql on errol with updates, won't help keeping IO low either!
19:17:46 <zere> then maybe worth having a 2nd server... but different in some way?
19:18:01 <pnorman> errol's db helped with debugging yevaud'd recent hiccup
19:18:11 <apmon> Perhaps we should have a separate discussion about how to move forward with the dev server?
19:18:17 <zere> rather than just dev2 - is it *for* something different, or for different projects?
19:18:23 <apmon> Including the various local chapters that have dev server infrastructure
19:18:48 <zere> yeah, we've overrun... i'll stick something in the plan about it anyhow. pull some reasonable looking figures out of the air.
19:18:58 <apmon> osm-de has some, osm-us, osm-fr and I think osm-ru as well
19:19:06 <zere> are they well-used?
19:19:20 <apmon> not sure? They all seem rather independent
19:19:28 <pnorman> Okay, moving forward. I can take on a check of if there are any blockers to generating OSMCoastline and look at that bug.
19:19:29 <zere> i think the osm-fr one is, iandees says the osm-us one perhaps not so much...
19:19:36 <apmon> and it might be more efficient to have a better coordination between the various resources
19:20:12 <zere> #action pnorman to check if there are any blockers to generating OSMCoastline and look at that bug.
19:20:16 <zere> pnorman: thanks :-)
19:20:41 <pnorman> someone to add errol's osm2pgsql replication delay to munin so we can track?
19:21:21 <zere> apmon: volunteering for that?
19:21:32 <pnorman> zere: needs to be someone with rights to change munin configs
19:21:42 <TomH> well no
19:21:49 <TomH> first somebody need to write a munin plugin
19:21:52 <apmon> it should just be an extra line in the munin conf files. So I suspect someone with admin rights needs to do that
19:21:54 <TomH> then they can ask me to deploy i
19:22:02 <pnorman> TomH: the same one as yevaud should work
19:22:16 <TomH> oh sorry I thought we were talking about yevaud
19:22:41 <TomH> what are we talking about then?
19:22:54 <pnorman> errol's osm2pgsql database's replication delay
19:23:30 <TomH> hmm... is that intended as a long term thing though? thought it was just an experiment?
19:23:50 <TomH> anyway I don't really want to be setting up monitoring on user-run things
19:24:02 <pnorman> well, I think part of the experiment was to see if it could keep up
19:24:51 <zere> i don't think it was an experiment - i thought it was intended for other people on errol to use in their own things, rather than need people to set it up themselves, etc...
19:25:24 <apmon> It probably was kind of both. The intention was to have it long term available if feasible.
19:25:43 <apmon> But setting up as user-run, was kind of an experiment or proof of concept
19:25:59 <pnorman> A thought to leave people with: Recently a way that crossed most of the globe brought yevaud to its knees with 100% of CPU going to renderd and z18 tiles taking 30+ seconds to render. We're not going to solve the cause here since we're running over, but its something to be aware of.
19:26:30 <zere> yup. i'll try and remember to start with that next time.
19:26:33 <zere> speaking of
19:26:47 <pnorman> time zones + DST suck.
19:28:16 <zere> next time will be the 2013-04-01: so it'll be the in-person meeting i've been telling you all about for months. you've all got your plane tickets, right?
19:28:36 <apmon> in person? where?
19:28:46 <pnorman> zere: where do we submit the recipts?
19:29:47 <zere> apmon: http://osm.org/go/bbWhEbYqW-- - it's the closest land to the center of the average attendees
19:30:15 <pnorman> and it has an airport
19:30:46 <apmon> I have the vague feeling it is only closest in geographic terms, not it terms of travel time or travel expenses... ;-)
19:31:23 <pnorman> well we're a geographic database, not a travel time or travel expense database
19:31:52 <zere> on 04-01, we can be any kind of database we like ;-)
19:32:23 <apmon> yeah, we should make the budget on that day
19:32:38 <apmon> and see if we can get it approved
19:32:48 <zere> 12 servers, you say?
19:32:56 <zere> better make it 24, just in case
19:33:17 <apmon> then we might be able to compete with wikipedia...
19:34:00 <pnorman> anyways, time on the next raid?
19:34:19 <zere> on a serious note, though - DST begins in europe between now and next meeting. do we want to change the time? in europe, the meeting will be 1 hour "later" than it is at the moment. in north america i think this has already happened.
19:34:36 <pnorman> I wouldn't mind the meeting going back to 10 AM local time
19:35:15 <zere> i have no preferance. - happy with 6pm or 7pm (BST)
19:38:02 <pnorman> 10 AM local is 6 PM BST
19:42:21 <apmon> anything else for todays meeting?
19:45:20 <zere> nope. i guess there's not much interest in the time, so we might as well make it 5pm GMT / 6pm BST / 10am (LWPIST - "local where pnorman is, summer time")
19:45:35 <zere> #agreed next meeting 2013-04-01 17:00 UTC
19:45:47 <zere> thanks to everyone for coming :-)