Attendees
| IRC nick
|
Real name
|
| apmon |
Kai Krueger
|
| emacsen |
Serge Wroclawski
|
| RichardF |
Richard Fairhurst
|
| rweait1 |
Richard Weait
|
| SteveSn_ |
Steve Singer
|
| TomH |
Tom Hughes
|
| zere |
Matt Amos
|
Summary
- TomH has produced a rails3 branch which brings us up-to-date with teh latest (?) version of Rails. There are some column name changes and the code could do with some testing; volunteers wanted.
- TomH has produced patches for cgimap and planetdump, but Osmosis will need to be updated to match.
- apmon to contact talk-de w.r.t perceived development issues and how we can help.
- RichardF suggested that a major barrier to entry is the sheer size of OSM data - even the extracts which are offered - and suggested tiled vector data downloads as a solution.
- SteveSn asked whether we had any information from downstream data users about what's stopping them setting up their own tile servers.
- RichardF suggested the documentation on the wiki [1] was not exactly straightforward. zere volunteered to clean up / reorganise them for optimal first-time viewing.
IRC Log
18:01 < RichardF> hello!
18:01 <@zere> minutes of the last meeting - let me know if anything is objectionable: http://www.osmfoundation.org/wiki/Working_Group_Minutes/EWG_2011-09-26
18:03 -!- rweait1 [~nerd@bas8-toronto01-1177860160.dsl.bell.ca] has joined #osm-ewg
18:03 -!- Firefishy [~openstree@proxy.wwood.co.uk] has joined #osm-ewg
18:04 <@zere> good afternoon, gentlemen.
18:04 -!- emacsen_ [~serge@205.201.255.5] has joined #osm-ewg
18:04 < emacsen_> appologies. The net here is incredibly unreliable
18:04 < emacsen_> emacsen, you got to go... sorry bud
18:04 -!- emacsen is now known as Guest12398
18:04 -!- emacsen_ is now known as emacsen
18:05 <@zere> there weren't many actions from the last meeting. one on apmon to send a message to talk-de about what is the most difficult thing with development. apmon?
18:06 -!- Guest12398 [~serge@205.201.255.5] has quit [Ping timeout: 480 seconds]
18:08 <@zere> ok, while apmon is typing his epic response, i sent out (admittedly at the 11th hour) a very short agenda for this meeting to dev@. i probably still need to publicise more.
18:08 < TomH> I'd also like to draw everybody's attention to my post about rails 3 and ask for people to try and do some testing
18:09 <@zere> TomH: this one? http://lists.openstreetmap.org/pipermail/rails-dev/2011-September/000057.html
18:09 < TomH> well not so much that as the one I sent just before
18:09 < TomH> though opinions on that are solicited as well
18:09 < TomH> http://lists.openstreetmap.org/pipermail/rails-dev/2011-September/000056.html
18:09 < TomH> is the other one
18:10 <@zere> aha... threaded view confused me ;-)
18:10 <@zere> everyone happy with giving TomH's rails3 branch a go? kick the tyres, so to speak?
18:11 < TomH> deploying it will need a slightly coordinated effort due to me having to change some field names
18:11 < TomH> I have updates for cgimap and planetdump but omsosis will need work as well
18:11 < apmon> zere: Unfortunately I didn't get around to doing it. So I'll try and do it this week
18:13 <@zere> TomH: what are the changes to 'core' tables like users/nodes/changesets/ etc...?
18:14 <@zere> apmon: no worries. would be good to engage a little more with the german dev community...
18:15 < TomH> zere: Basically I had to bite the bullet and add proper primary keys to all the models, which in turn required renaming the "id" column in a number of tables as "id" is special in rails and it gets very confused if you have id as part of a composite key
18:15 -!- emacsen [~serge@205.201.255.5] has quit [Read error: Operation timed out]
18:15 < TomH> so id has become node_id/way_id etc in a number of tables
18:17 < TomH> https://github.com/tomhughes/openstreetmap-website/blob/rails3/db/migrate/20110925112722_rename_ids.rb
18:17 < TomH> is the changes
18:19 <@zere> cool.
18:19 <@zere> so the general topic for this meeting is supposed to be "what's the biggest single thing stopping you developing with OSM data?"
18:19 <@zere> anyone want to start?
18:20 -!- emacsen [~serge@173-203-96-137.static.cloud-ips.com] has joined #osm-ewg
18:20 < emacsen> Okay, sorry about that. I'm trying through a proxy. This may actually be more stable
18:20 < RichardF> for me: sheer size of it
18:20 < emacsen> If I drop off again, forgive me
18:23 <@zere> RichardF: are the geofabrik / cloudmade / stamen extracts still too large to work with?
18:24 * Firefishy has to unfortunately drop off, cheers.
18:24 < RichardF> zere: geofabrik extracts are still pretty big. Mike Migurski's ones are more manageable but only for a few cities, of course
18:24 -!- Firefishy [~openstree@proxy.wwood.co.uk] has quit []
18:25 <@zere> so there's a gap, you think, between the area downloadable with a map call and the smallest / most relevant extract?
18:26 < RichardF> personally I think there might be, yes. I've wondered whether it might be worthwhile for OSM to provide weekly 1^P^P^P-degree tiles. but I'd be interested to hear others' views.
18:26 < apmon> RichardF: For what kind of application? And why is it to large?
18:26 < emacsen> what is 1^P^P^P? I don't see those last three characters
18:26 < apmon> I.e. what resource do the geofabrik extrcts exced?
18:27 < RichardF> apmon: for me, the UK one is a lot of time (i.e. slow) and space to play around with easily. others' expectations may differ :)
18:28 < RichardF> bearing in mind the raster tile conversation that's happening simultaneously, I do also wonder whether if OSM were to distribute small tiles, map app vendors would find it easier to work with vector data than they do now.
18:28 < apmon> Yes, I guess the UK one has become to big. Germany has sub extracts on the state level, which are mostly managable
18:29 < apmon> RichardF: Is serving vector tiles to clients any less resource intense than serving map tiles to clients?
18:29 < RichardF> apmon: it's easier for app providers to mirror
18:30 < emacsen> do we have a sane way to serve vector data <z12?
18:30 < emacsen> or, hell, z14
18:30 < SteveSn_> has someone tried talking to some of the map vendors using our tile-server to find out what the road blocks are for them to run there own?
18:30 < RichardF> on another "barrier to developing" - I'd be interested to know what people's thoughts are on whether there are good libraries for working with OSM data, or whether they're needed
18:31 < RichardF> I was slightly surprised recently to start using Jochen's osmlib (Ruby), which is good, but pretty raw and unpolished - is the situation the same for other languages? would it help?
18:31 <@zere> RichardF: what do you think makes it easier for app providers to mirror vector data?
18:32 < emacsen> RichardF, there are a few ones for python, nothing so unified though, honestly. There are mostly xml parsers and a few pbf parsers
18:33 < RichardF> zere: smaller, more efficient data. at least from my point of view
18:33 <@zere> i dunno about that - is it really smaller data, given that there's no filtering done?
18:34 < RichardF> it seems so to me. but all I'm telling you about is what scares me when I'm trying to do stuff with OSM. I'm not saying it's necessarsily rational :)
18:38 <@zere> let's put it to the test...
18:38 < apmon> Overall, I would agree though that size is one of the biggest hindrances of working with OSM. And without the geofabrik and other extracts it would be near impossible
18:39 < RichardF> a third thing to offer up: documentation. I _think_ I know which blogs I'd search and which pages I'd look at if I wanted to set up (say) an osm2pgsql and Mapnik setup, but it's not consistently in one place
18:44 <@zere> hmm, seems RichardF is right - the OSM data is smaller by a factor of 10!
18:44 <@zere> (when gzipped)
18:44 < emacsen> yeah and with pbf, it's even greater reductions.
18:45 <@zere> well, sure. but then i'd have to go and pngcrush all the images too, just to make sure it's a fair test ;-)
18:46 < emacsen> okok
18:47 <@zere> so... sounds like it'd be a good idea to see if geofabrik / cloudmade / someone wants to start doing some sort of tiled extract.
18:48 <@zere> about the documentation, it sounds like this page http://wiki.openstreetmap.org/wiki/Mapnik isn't very helpful, then?
18:48 <@zere> is it that it needs updating, or just that it's too long and complex?
18:49 < apmon> I need to reread it again, but I think it has too many things thrown in onto a single page
18:49 <@zere> and how soon can we replace it with apmon's "sudo apt-get install mod_tile osm2pgsql"? ;-)
18:50 * apmon needs to get back to that...
18:50 < apmon> If I get around to it, perhaps in a week or two
18:50 < apmon> but it is always dangerous to promise things
18:51 <@zere> so perhaps that wiki page needs to be broken up into parts for the major headings it currently has, and re-written (perhaps) to be more linear?
18:51 < apmon> I went back to parallelising osm2pgsql to speed it up some more
18:51 <@zere> RichardF: would that address your issue?
18:51 <@zere> apmon: diff updates?
18:51 < apmon> both
18:52 < apmon> It parallelizes the "going over pending relations / ways" part in both imports and diffs
18:52 < apmon> Using forking rather than threads as Fredrick suggested made live much simpler
18:53 <@zere> cool
18:53 < apmon> Now I need to test it, to see if it actually produces correct output or creates artifacts in some way
18:53 <@zere> run the unit te... oops ;-)
18:53 < RichardF> zere: that would help. Richard W's Mapnik tutorials are a nice way to do it - lots of screenshots etc. to demonstrate what you're getting out of the process. Not sure whether it would help to collate/reprint those in central OSM documentation or if just some links might help...
18:54 < emacsen> I think the other thing is a question of goal. RWeait's articles are howtos, the Wiki seems to be about "Mapnik" in the most general sense. I suspect 90% of users want the howto
18:54 <@zere> RichardF: imho, linking is better, but there seems to be more information on the wiki page, and more translations. so probably worthwhile keeping the content that's there and reorganising it to make it easier to follow.
18:55 <@zere> emacsen: yeah, 90% of people want the big link at the top which says "howto is this way ==>" ;-)
18:55 * RichardF nods.
18:55 < emacsen> there's a big link at the top?
18:56 < emacsen> I'm sorry am I blind? I don't see it
18:56 <@zere> in my imaginary land where the documentation is better, there is :-)
18:56 <@zere> also there is http://wiki.openstreetmap.org/wiki/HowTo_mod_tile
18:56 <@zere> so some cleanup is required.
18:57 <@zere> any volunteers?
19:00 <@zere> c'mon, RichardF, you know you want to.
19:01 * apmon should rather work on getting documentation needs down to "apt-get install osm-tileserver"... (at least for ubuntu)
19:02 <@zere> well, that's kinda a problem too. large installation docs for windows / mac os x / ubuntu / centos / etc...
19:03 <@zere> perhaps this is a good time to mention VMs and poke emacsen? ;-)
19:03 < emacsen> Yeah I'm poked... Okay, so I dropped the ball here entirely. I need to essentially start from scratch. Which reminds me... TomH if Rails3 is the current target, should that be what I focus on?
19:03 < emacsen> or is rails not the objective at all, since I'm starting anew
19:03 < apmon> Perhaps we should convince SteveC to write some documentation on how to do things on windows ;-)
19:04 < emacsen> apmon, be nice... :)
19:04 <@zere> emacsen: i think rails is the first target. and when apmon has his osm-tileserver virtual working then we'll be golden.
19:04 < emacsen> ok
19:05 < emacsen> Since my schedule is so wonky, I'll post something by Sunday on my progress whatever that is
19:05 <@zere> cool, thanks :-)
19:05 < emacsen> I'll be mostly free part of wed on a train
19:05 < emacsen> so I'll have a few hours with few excuses
19:06 <@zere> sounds like a plan.
19:07 <@zere> so to finish up the meeting we just need RichardF to volunteer to clean up the wiki, or someone else to take a bullet for him...
19:07 <@zere> any takers?
19:07 <@zere> rweait1, perhaps?
19:09 <@zere> rweait1: por favor, limpiar la Wiki?
19:10 < emacsen> hold on, I'll do it the American way
19:10 < emacsen> RWEAIT1, FIX... THE... WIKI....
19:10 < rweait1> Nice.
19:10 < rweait1> I'm happy to maintain my howto.
19:11 <@zere> in IRC i can't see you waving the money in one hand and the speedial for the INS in the other.
19:12 <@zere> ok. since we have no volunteers, i will fall on this landmine myself.
19:12 <@zere> *clickedy* *click*
19:12 <@zere> the wiki is clean...
19:13 <@zere> ok. thanks to everyone for coming! see you next week :-)