Attendees
| IRC nick
|
Real name
|
| pnorman |
Paul Norman
|
| shaunmcdonald |
Shaun McDonald
|
| zere |
Matt Amos
|
Summary
- API 0.7
- ACTION pending on zere to set up api07.apis.dev.openstreetmap.org
- gravitystorm mentioned that it would be good to have cgimap as a gem, so it can take over request handling within rails_port too (although in production it would probably be bypassed)
- TODO in cgimap (the last few commits on the 'development branch' were ground-work for history support https://github.com/zerebubuth/openstreetmap-cgimap/commits/library_split)
- still need full API coverage in 0.6/0.7 - most API calls will be the same, of course.
- need code for an OAuth provider to use in cgimap
- need to test [node|way|relation]/relations and node/ways
- need loads more tests.
IRC Log
17:33:30 <zere> minutes of the last meeting: http://www.osmfoundation.org/wiki/Working_Group_Minutes/EWG_2014-06-30 please let me know if anything needs changing
17:34:28 <zere> updates on actions: i had an action to set up an api0.7 branch, but have not done so. apologies. i was hoping to have more time last week, but it wasn't to be.
17:34:37 <zere> i'll try to get it done by next meeting.
17:34:44 <pnorman> isn't that git checkout -b api0.7?
17:35:10 <shaunmcdonald> yup then a push, and using a new enough client does all the right setup for you.
17:35:32 <zere> that was more about getting it on apis.dev, i think.
17:36:03 <zere> since, as you point out as if i'm an idiot, setting up local or remote branches in a git repo is really easy.
17:36:06 <shaunmcdonald> I’ve only read the summary of the last meeting. I’m slightly concerned about code going into cgimap and not rails_port.
17:36:30 <pnorman> zere: well, I was more wondering what the action item consisted of
17:36:51 <shaunmcdonald> zere: fair point, it wasn’t clear in the summary that was needed too.
17:37:35 <pnorman> I think there's a desire to seperate the API code from the website code where possible
17:38:31 <pnorman> Also, experience shows that cgimap can be an order of magnitude faster than the rails port
17:39:08 <zere> ok, hopefully that action in the previous minutes is a bit more detailed now
17:41:17 <zere> shaunmcdonald: what's your concern about code going into cgimap and not rails_port?
17:42:25 <shaunmcdonald> pnorman: I’m just wondering how that will affect development things, and ease of setup, as up until now we’ve primarily had c code to duplicate rails functionality so that the c code is optional.
17:42:42 <pnorman> note: cgimap is C++
17:43:32 <pnorman> Personally I don't object to the code going only in cgimap, but our past practice has been to duplicate the rails port functionality in cgimap, this would be a change
17:44:02 <zere> well, c or c++ doesn't really make a difference to the concern, i think.
17:44:23 <shaunmcdonald> C or C++ doesn’t make any difference in this case.
17:44:48 <zere> gravitystorm mentioned to me that i should try to make cgimap a gem, so it can take over request handling within rails_port too (although in production we would probably bypass that)
17:45:35 <pnorman> Ya, just noting it for the minutes. Fwiw, I found cgimap deploy easier than rails deploy, but harder than bundle exec rails server
17:45:37 <zere> just so that installing a working server was still "bundle install", rather than some complex set of instructions about cgimap and so forth
17:46:46 <zere> clearly installing one bit of software (and having the other handled by bundler) is better than two, but does it address your concern, shaunmcdonald?
17:47:08 <zere> or is the concern also about whether the code can be changed in rails_port?
17:48:37 <shaunmcdonald> yup, like the idea of a gem to make development easier would be good. Of course if we do diverge, we would need to make the rails code return an error explaining how to setup the cgimap if it isn’t running.
17:48:37 <pnorman> from a maintainability point of view, the code duplication is the same, particularly when the rails code never gets tested in production. I'm recalling the differences between the two I found for GSOC
17:48:47 <pnorman> s/is the same/is a pain/
17:49:15 <shaunmcdonald> do all the api dev instances use cgimap now?
17:49:28 <pnorman> Do *any*?
17:50:08 <shaunmcdonald> Need to have more clone projects that use the rails side ;-)
17:52:15 <zere> indeed, i can't see any chef rules for cgimap used on errol.
17:52:33 <zere> nor do any cgimap processes seem to be running
17:53:33 <zere> but yes - the plan (long term, perhaps not in time for 0.7) is to make the API separate from the website, so the two can be run (managed, fixed, re-written) separately
17:54:21 <pnorman> For 95% of what the website does, it could rely on the public API calls. See http://osmlab.github.io/osm-deep-history/ as an example of the idea. But this is probably longer term
17:55:24 <shaunmcdonald> for clarity which methods do you mean when you say API, is it everything under /api or just the read requests?
17:57:52 <pnorman> I'd *like* to see it all, but I see the read requests as more likely.
18:00:17 <zere> well... it's unlikey that we'll support gpx traces any time soon. in fact, i'd prefer to see that as an entirely separate application.
18:00:33 <zere> also, the user preferences would seem to fit better with the rest of the website.
18:01:19 <shaunmcdonald> Also getting into auth logic with the write requests
18:01:27 <zere> finally, i'd like to see some of those API calls not make it to 0.7 (/changes, /**/search, /amf/**, etc...)
18:01:38 <pnorman> In an ideal world I think someone should be able to duplicate all unlogged in non-user parts of osm.org by pulling against the API, and all data uploads.
18:01:54 <pnorman> zere: and the single-object uploads
18:02:27 <shaunmcdonald> are there stats on how much the single object uploads are used.
18:03:20 <pnorman> almost not at all. I think matt posted stats in one of his talks
18:05:33 <pnorman> I think perhaps some of the low-level command-line tools do, but certainly none of the big editors.
18:06:54 <zere> the stat was "99% of users use changeset/upload"
18:07:00 <pnorman> What can we take away as items to do from this?
18:07:58 <zere> we still need full API coverage in 0.6/0.7 - most API calls will be the same, of course.
18:08:31 <zere> we need code for an OAuth provider to use in cgimap
18:09:01 <pnorman> hmm yes, and we need to test [node|way|relation]/relations and node/ways
18:10:05 <zere> the last few commits on the 'development branch' were ground-work for history support https://github.com/zerebubuth/openstreetmap-cgimap/commits/library_split
18:10:42 <zere> and yes, we need loads more tests.
18:11:02 <zere> again, probably better to look at the library_split branch - tests run much quicker there since it's mocking out the database
18:17:04 <zere> hmmm... we seem to be running out of steam
18:17:05 <pnorman> I need to go to the bathroom, but for AOB, DWG/LWG would like https://trac.openstreetmap.org/ticket/5024 resolved. I tried tackling it, but ran into limitations on my ruby skills, or complete lack thereof
18:17:21 <zere> #topic AoB
18:18:19 <zere> that's the editor imagery blacklist. should be pretty easy.
18:18:25 <zere> any volunteers?
18:20:33 <zere> seems like the proposal is to add a blacklist *and* a whitelist? this is pretty confusing, since i've not been following the back-story...
18:20:43 <shaunmcdonald> I like the idea of an admin interface for adding the imagary blacklist. I presume it would be best to store the entries in the DB so that they can be easily updated, or a simple config file that is read in?
18:21:27 <zere> an admin interface would add more work, but potentially better for other people who might be using rails_port.
18:21:40 <zere> i was thinking just a YAML file in config/ would work fine...
18:22:05 <shaunmcdonald> I’d say blacklist only
18:22:47 <shaunmcdonald> I was thinking just a plain file, with one entry per line, with each entry being a regex. Not sure what advantage a yaml file would bring.
18:22:57 <zere> for example those guys at opengeofiction, presumably they potentially have their own hand-drawn maps to trace over.
18:23:55 <zere> YAML is slightly nicer if only because it supports a bunch of these types natively, is readable in code, and also supports comments in the source.
18:24:07 <zere> but to be honest, i'm bike-shedding a bit here
18:24:34 <zere> i think it's more important to have something than for it to be whatever i would have written, given my non-existant free time ;-)
18:24:46 <zere> shaunmcdonald: feel like having a go at it?
18:25:10 <shaunmcdonald> reading the JOSM commits, it seem to be only a blacklist that they’ve got.
18:26:34 <shaunmcdonald> yup can give it a shot, was just exploring a bit about how it should be done before volunteering. I’m thinking a config the same way as application.yml, or maybe even putting the options into application.yml?
18:27:04 <zere> right, and i'm not sure what happens if you have a whitelist and a blacklist - it's allowed if it's on the whitelist and not if it's on the blacklist and if it's on neither than it just...? downloads only half the tiles?
18:27:54 <pnorman> It's solely a blacklist - there is no API whitelist
18:28:27 <zere> so it seems a good place to start would be a blacklist, something in /capabilities which advertises it an a URL to download the regexes in (say) some XML list. does JOSM already have a standard format for that list?
18:29:03 <pnorman> zere: yes - the way JOSM implements it is to shoe-horn the blacklist into the capabilities doc after downloading
18:29:05 <shaunmcdonald> JOSM already reads it in from the capabilities, if available
18:29:37 <pnorman> http://josm.openstreetmap.de/browser/josm/trunk/src/org/openstreetmap/josm/io/Capabilities.java#L15
18:30:33 <pnorman> <osm><policy><imagery><blacklist regex="quoting hell..."/><blacklist .../></imagery></policy></osm>
18:30:36 <zere> cool, that seems pretty reasonable... shaunmcdonald, what do you think?
18:31:37 <shaunmcdonald> I’m happy with it being put in there
18:32:12 <shaunmcdonald> Is everyone else happy for me to store them in the config/application.yml file?
18:34:12 <zere> yeah, that seems like a good place for it
18:35:24 <zere> okay, if we're all done with that...
18:35:24 <shaunmcdonald> cool, I can have a stab at it in the next few weeks.
18:35:33 <zere> shaunmcdonald: awesome, thanks :-)
18:36:00 <zere> final item - i can't make the next 2 meetings. does anyone want to volunteer to chair?
18:37:38 <zere> have a think about it - i'll pester gravitystorm to see if he'd be happy to step in.
18:37:48 <zere> thanks to everyone for coming :-)