Working Group Minutes/EWG 2014-01-27

From OpenStreetMap Foundation

Attendees

IRC nick Real name
apmon_ Kai Krueger
pnorman Paul Norman
RichardF Richard Fairhurst
shaunmcdonald Shaun McDonald
TomH Tom Hughes
zere Matt Amos

Summary

  • Routing branch
    • RichardF has done some great work, and a running instance is on the dev server.
    • Remaining work items:
      • i18n - many backends support this, but the frontend needs to as well.
      • permalinking - so people can share links to routes.
      • backends - need to contact backend providers to check they're OK with the potential load.
    • Also: a "loading" message / spinner, and an error handler.
    • Issues with old IE versions, may need changes to support XDomainRequest.
  • Blogs
    • shaunmcdonald made a bit of progress on the update to blogs.osm.org: Got the code up, though needs some more work before it can go live, partly as the the RSS feed templates aren't ready yet code on github.
  • Osm2pgsql threading
    • pnorman found differences in the imported data between the threading branch and master. The issue seems to be with a single section of patch, and can be tested to check.

IRC Log

17:33:02 <zere> minutes of the last meeting: http://www.osmfoundation.org/wiki/Working_Group_Minutes/EWG_2014-01-20 please let me know if there are any issues.
17:34:01 <zere> actions from the previous meeting: i had one held over to set up an EWG alias/list for people to contact when they aren't able to attend IRC meetings. that's done; engineering@osmfoundation.org. some of you should have received emails.
17:34:34 <zere> other than that, i don't think there were any.
17:35:12 <zere> last meeting we were talking about the routing branch, and i think RichardF has been hard at work on that all week. RichardF: how goes it? ;-)
17:35:40 <RichardF> yep :)
17:35:46 <apmon_> Yes, it looks like some really nice work RichardF has done!
17:35:54 <RichardF> code is at https://github.com/systemed/openstreetmap-website/tree/routing
17:36:02 <shaunmcdonald> I've made a bit of progress on the update to blogs.osm.org. Got the code up, though needs some more work before it can go live, partly as the the RSS feed templates aren't ready yet. https://github.com/smsm1/blogs.osm.org
17:36:06 <RichardF> running instance is at http://jsrouting.apis.dev.openstreetmap.org/
17:36:16 <RichardF> so the main things still to do are:
17:36:34 <apmon_> I had a brief look at the i18n stuff for routing
17:36:42 <RichardF> a) permalinks (e.g. osm.org/route?from=51.5,2.1&to=51.6,2.2&router=osrm_car)
17:36:47 <RichardF> b) i18n, like apmon_ says :)
17:36:57 <apmon_> it looks like there are not yet many strings to translate. So it probably is fine to leave it to the end
17:37:02 <RichardF> c) minor matter of actually having an OSMF-hosted routing server
17:37:12 <apmon_> The main question I could see is how to deal with the translation of the route instructions
17:37:34 <apmon_> i.e. should the routing backend do that translation, or do all instructions get "normalized" at the frontend
17:37:40 <RichardF> apmon_: routing backend without a doubt
17:38:01 <RichardF> apmon_: the approach is very much that we just take the instructions provided by the backend, and present them without much faffing about
17:38:07 <apmon_> So then the question is how many of the routing backends do support giving instructions in different languages?
17:39:08 <apmon_> OK.
17:39:09 <RichardF> certainly MQ and CM do
17:39:43 <apmon_> I can probably do a pull request for the i18n stuff later in the week if you want. As said, it looks pretty minor so far as to the number of strings involved
17:39:53 <RichardF> apmon_: that'd be brilliant - yes please
17:40:00 <apmon_> and the rails port stuff looks like you have already done the i18n stuff :-)
17:41:09 <apmon_> Question for TomH: If you are doing i18n in a js.erb template. Should one use the javascript i18n functions, or the rails i18n functions?
17:41:18 <zere> i was wondering about that - is it easy for rails to export the translations as json, then you refer to them by their "t()" name?
17:41:45 <apmon_> Yes, that is how I believe it is normally done for js code
17:42:11 <apmon_> but if the js code is dynamic anyway (.erb template), one could skip that step
17:42:28 <TomH> yes
17:42:46 <TomH> apmon_: well you probably shouldn't be using js.erb in most cases
17:42:58 <TomH> I know RichardF's code is, but that is fixable
17:43:30 <zere> you mean, pull out the bits which don't need to be in the erb as static .js files?
17:44:04 <shaunmcdonald> You also get stung by the rails optimising the assets pipeline, as it's best to have a single js file for everyone.
17:44:06 <apmon_> Yes, the more one can do in static files the better. For performance reasons and cachability
17:44:07 <TomH> well last I looked it was only the nominatim url that it was interpolating? and that can just go in osm.js.erb with the other settings from application.yml
17:44:44 <TomH> oh that's true, you can't do translations that way anyway as the erb is only evaluated once when the assets are compiled
17:45:04 <shaunmcdonald> Can use data attributes in the html to store any data that needs to change.
17:45:06 <apmon_> Well, that answers the question then... ;-)
17:45:28 <TomH> but don't worry about the details - we can handle that in review
17:45:38 <zere> so the rails page will need to dynamically include different language-specific js files which are built as assets...
17:46:04 <TomH> seriously, js translations is a solved problem
17:46:21 <TomH> just use I18n.t in the js
17:46:37 <apmon_> yes, it is all nice in that plugin and easy to use
17:46:41 <TomH> and make sure config/i18n-js.yml is exporting them to js
17:48:29 <RichardF> ok - other than i18n, it would be good to hear of anything that the routing branch would absolutely have to do for a first release, and doesn't currently
17:49:24 <RichardF> I don't intend to build a fully-featured Google-like UI with infinite via points etc. (not for the first release, anyway ;) )
17:49:40 <RichardF> we can do the 'nice-to-haves' once it's up and running
17:49:51 <shaunmcdonald> RichardF: Nah you'll just divert them to cycle.travel ;-)
17:49:55 <RichardF> shaunmcdonald: heheh
17:50:15 <apmon_> Does the Car(OSRM) one currently work?
17:50:22 <RichardF> should do
17:50:25 <zere> yeah... well, worksforme
17:50:30 <zere> is it EU-only?
17:51:08 <apmon_> doesn't for me
17:51:41 * RichardF just routed from Portland to San Francisco using Car (OSRM)
17:51:47 <apmon_> Oh, I am getting the forbidden, your IP has been blocked...
17:51:52 * RichardF laughs
17:51:52 <RichardF> ah
17:51:55 <zere> some of the providers are... not as fast as the others. is it easy to add a spinner where "directions" is, saying "calculating route" or something?
17:52:07 <RichardF> zere: +1, yes, definitely ought to do that
17:52:44 <zere> i guess an error handler as well, for when one's IP gets blocked ;-)
17:53:01 <RichardF> one other issue is that it doesn't autozoom to the calculated route
17:53:04 <apmon_> :-) Yes that would be helpful.
17:54:17 <zere> RichardF: are you doing this at https://github.com/systemed/openstreetmap-website on your master branch?
17:54:38 <apmon_> Other nice to have features might be to specify the time it takes in addition to the distance on the instructions
17:55:10 <zere> i think that's going to depend on whether the backend returns that information. i think most do
17:55:58 <zere> actually, sorry, RichardF already said on his "routing" branch. oops ;-)
17:56:45 <apmon_> It can be pretty helpful to see if things like maxspeed are wildely off on a route
17:58:01 <apmon_> Another question / nice to have feature could be if the main search dialog should support "A to B" syntax
17:58:25 <zere> true. i think it's likely (i hope, anyway) that when this is released it'll encourage people to make "debugging" routers, just like we have "debugging" map styles - which show/highlight particular features.
17:59:09 <zere> well, it would be nice, but i've heard i18n NLP is a complex problem ;-)
17:59:36 <apmon_> But imho the sooner we can get this deployed the better.
18:00:27 <RichardF> ok. there's a few little things I can tackle this week, then, but other than that, I guess it's over to OWG to think about an OSMF-owned routing server...? (or would we be happy to launch if entirely dependent on 3rd-party engines?)
18:01:14 <apmon_> OSMF would definately have to talk to all of the 3rd parties involved
18:02:02 <zere> speaking personally, i'd be happy to launch on 3rd party engines. there's a plan for OSMF-owned routing servers, but i don't see the rush as long as the 3rd parties involved are happy too.
18:02:43 <apmon_> Which of the backends are opensource? OSRM and Graphhopper?
18:02:52 <RichardF> yep
18:03:20 <RichardF> we could run against a Gosmore backend too, of course, but on a cursory examination I couldn't see a JSONP callback at YOURS
18:03:46 <TomH> urgh we're not relying on that horrid hack are we?
18:04:27 <zere> is there a non-hacky way of doing it which works with IE cross-domain?
18:04:58 <RichardF> TomH: only insofar as I wanted to get something up and running without having to spend n days nagging people to add CORS headers!
18:05:10 <TomH> what versions of IE do CORS?
18:06:09 <RichardF> http://caniuse.com/cors "Supported somewhat in IE8 and IE9 using the XDomainRequest object (but has limitations)"
18:08:17 * pnorman waves, off the phone with IT
18:09:03 <pnorman> When we're done with routing, osm2pgsql threading needs some discussion
18:09:33 <zere> RichardF: http://stackoverflow.com/a/11517652 - but do we pull in jquery already?
18:09:43 <RichardF> zere: yeah, we do
18:09:47 <RichardF> that's worth knowing, thanks
18:10:22 <zere> although there seem to be conflicting comments about whether it actually works or not.
18:12:12 <TomH> I think that is very confused....
18:12:27 <TomH> $.support is a set of readable attributes not something you should be writing
18:12:33 <TomH> and it's also internal only
18:12:37 <TomH> http://api.jquery.com/jQuery.support/
18:12:53 <TomH> I suspect it is automatic and $.support.cors tells you if the browser supports it
18:13:40 <zere> haha, oops. then i guess there's some fiddling needed to make requests via XDR when using a crappy browser.
18:16:41 <zere> ok, cool. let's talk about osm2pgsql threading :-)
18:16:49 <zere> #topic osm2pgsql threading
18:16:57 <zere> pnorman: how goes it?
18:17:43 <pnorman> https://github.com/openstreetmap/osm2pgsql/pull/110#issuecomment-32048287
18:17:57 <pnorman> Basically, we've got different results in the lines and roads tables
18:20:29 <apmon_> pnorman, could I get access to the benchmarking server when you are done, to see if I can pinpoint more what is going on.
18:20:43 <pnorman> done? :P
18:20:46 <apmon_> I'll have to double check, but I think I couldn't reproduce this with smaller examples
18:24:37 <zere> pnorman: https://github.com/openstreetmap/osm2pgsql/compare/9fc4fd49789067951bb7c309756310588fae96c4...a5755430392af3459d06ede91e9498b10bd18ae1#diff-8647d42f728e3cd81a88ef0c2b3151ecL1726 ?
18:26:01 <pnorman> yep, that's the compare
18:27:30 <zere> well, specifically L1726 of output-pgsql.c
18:28:29 <zere> moving the delete inside the if(){} block. that would execute the delete less often and could plausibly lead to the threaded/normal results you're seeing.
18:29:05 <pnorman> well doesn't that only impact the roads table, not the lines table, and we see it on both?
18:30:32 <zere> yes, you're right
18:30:39 <zere> how about L878?
18:31:14 <pnorman> I think that's what I had identified as a preliminary cause
18:31:52 <zere> assuming Options->append == 0, then it's not doing that delete any more. from the looks of things, that might not be a bug?
18:32:16 <apmon_> line 1726 should not be a problem. The if statement (should) test if the way is in the lines table. If it isn't in the lines table, it shouldn't be in the roads table either, as that is a strict subset of lines.
18:32:33 <apmon_> So there isn't a need to do another SQL round trip to delete something that isn't in the db
18:34:29 <zere> sure, i suppose reversing the L878 patch and doing a re-import would let you find out for sure.
18:34:44 <apmon_> If we aren't in append mode, I believe those members never get added in the first place. But it is something to verify and make sure that is indeed the case
18:36:33 <zere> cool. is there anything else anyone wanted to discuss?
18:37:14 <apmon_> put from those lines, it is still the highest likely to be the culprit, so indeed just undoing that if statement and checking it out would be good
18:39:25 <pnorman> k, I'll test with https://github.com/openstreetmap/osm2pgsql/compare/9fc4fd49789067951bb7c309756310588fae96c4...a5755430392af3459d06ede91e9498b10bd18ae1#diff-8647d42f728e3cd81a88ef0c2b3151ecL878 reverted
18:40:13 <zere> anything else.... going twice?
18:41:23 <zere> ok. thank you all for coming, and i hope to see you next week! :-)
Retrieved from "https://osmfoundation.org/w/index.php?title=Working_Group_Minutes/EWG_2014-01-27&oldid=2497"