Attendees
| IRC nick
|
Real name
|
| apmon_p |
Kai Krueger
|
| gravitystorm |
Andy Allan
|
| iandees |
Ian Dees
|
| pnorman |
Paul Norman
|
| RichardF |
Richard Fairhurst
|
| TomH |
Tom Hughes
|
| zere |
Matt Amos
|
Summary
- Code4OSM [code4osm.org]
- zere announced this new site, and asked for help. Fork, edit & PR here.
- Hackday advice
- Some futher discussion of the hackpad doc [1], and it was thought reasonable.
- ACTION: gravitystorm to write it up as a PR for code4osm.
- osm2pgsql benchmarking
- pnorman asked for help with hardware, specifically a 4-8 core, 24GB RAM, 300-480GB SSD machine for benchmarking osm2pgsql parallel loads.
- zere offered to get a Hetzner machine.
IRC Log
17:30:45 <zere> welcome all to the exciting new time of 17:30 UTC ;-)
17:30:53 <zere> minutes of the previous meeting: http://www.osmfoundation.org/wiki/Working_Group_Minutes/EWG_2013-10-28
17:31:09 <zere> please let me know if any corrections are needed.
17:31:16 <zere> #topic previous actions
17:31:54 <zere> gravitystorm: anything on the previous events or OWL docs to report?
17:32:03 <gravitystorm> zere: no, sorry
17:32:24 <zere> no worries. i'm still on the hook for a blog post, which i really should get around to.
17:32:59 <zere> #topic code4osm
17:33:10 <zere> i'd like to announce http://code4osm.org/
17:33:28 <RichardF> \o/
17:33:44 <zere> which, at the moment, is a crappy bootstrap + ruby horror, the full "glory" of which can be seen here: https://github.com/zerebubuth/code4osm.org
17:33:53 <zere> which is why i'd love your help making it better!
17:34:35 <zere> it basically just checks out various projects and processes their .md files.
17:34:48 <zere> it could do with better styling, and working out how navigation is supposed to work.
17:35:04 <gravitystorm> zere: is roll-your-own vs jekyll a deliberate decision?
17:35:10 <zere> also, i think i broke some of the bootstrap code while i was stealing^W taking inspiration from it.
17:36:39 <zere> gravitystorm: yes. i've tried to use jekyll a billion times, and i've never got it to work the way i wanted it to work. in this case i just needed something to basically do "projects.each {|p| p.checkout; p.make_html_from_markdown}", and i couldn't figure out how to do that in jekyll
17:37:48 <gravitystorm> ah, I see - you're pulling content from the upstream repos rather than creating more content
17:38:34 <zere> mostly, yeah. i wanted this to not become a silo. ideally, each project would have and keep its own documentation.
17:39:05 <zere> this is just to bring them into one place, and add some structure and glue prose.
17:39:26 * pnorman waves
17:45:33 <zere> i shall take this silence as enthusiastic hacking on code4osm :-)
17:45:53 <zere> #topic hackday advice
17:45:55 <pnorman> you can take my silence as my brain still booting up
17:46:07 <zere> we were putting together https://hackpad.com/EWG-hack-day-recommendations-lh95anw01vN last time
17:46:17 <zere> i've tried to dump as much in there as i could think of
17:49:14 <pnorman> I'd say it looks reasonable. What do we do with it now?
17:49:42 <zere> cool. i guess we want to write it up somewhere on the wiki, and put some links in so that people can find it?
17:50:10 * zere makes a mental note to add a paragraph about it to the blog post he still has to write.
17:50:59 <zere> anyone feel like taking an action to make a wiki page with that content?
17:51:15 <gravitystorm> I'll make a pull request for code4osm :-)
17:51:40 <zere> ok, sure. that would be good also
17:52:13 <zere> #action gravitystorm to write up the hackpad hackday hackrecommendations for code4osm, and submit a PR.
17:52:18 <zere> gravitystorm: thanks :-)
17:52:33 <gravitystorm> more seriously, I think it's worth mentioning the difference between our-style hack days and the more common "challenge"-style hack days. I mean, in the publicity section make it clear that organisers need to set expectations
17:53:42 <zere> yes. i thought about adding that in the "organisation" section - you've got to make a decision pretty early between a "free form", "themed" and "competitive" hack day. they're completely different events to run.
17:54:40 <zere> but i couldn't think of a good way to write it. i've never organised anything but the "free form" style, with as little structure as possible.
17:56:22 <gravitystorm> ok, I'll have a think
17:58:42 <zere> is there any more we want to do on this?
17:59:06 <gravitystorm> not from my PoV
17:59:21 <zere> i wondered if there were cultural / national things which might need to be called out (i wouldn't know - i've only ever done these things in the UK)
18:00:00 <zere> are there things which people expect from an event in germany, france, USA, canada which we've not put on the list?
18:00:31 <zere> i was thinking about it particularly when writing: "aving an informal session after the hacking is over (e.g: dinner, drinks) is a good way for people to connect, and an opportunity to include people who might not have wanted to attend the main event. Make sure the venue is announced and, if necessary, booked well in advance of the date."
18:01:09 <zere> and not just cultural in the "'aving an informal session after the 'acking is over, innit" sense...
18:02:59 * pnorman is catching up on the osm-carto mergefest still :)
18:04:06 <pnorman> I scared him off!
18:04:51 <pnorman> I have a sort of list fo stuff I'd like to do for osm2pgsql benchmarking, but after getting my last EC2 bill, I'm not paying for it
18:07:01 <zere> pnorman: what sort of thing are you looking for?
18:07:09 <pnorman> With threading branch, how does way column correlation change with number processes? How does ORDER BY time change? CLUSTER time?
18:07:12 <pnorman> Is CLUSTER or ORDER BY faster?
18:07:13 <zere> #topic osm2pgsql benchmarking
18:07:14 <pnorman> How much does CLUSTER improve rendering anyways?
18:07:17 <pnorman> How fast does CLUSTER degrade?
18:07:19 <pnorman> plan: import, run diffs, benchmark, reindex, benchmark, cluster, benchmark
18:07:22 <pnorman> what performance slowdown is there from going to purely hstore?
18:07:24 <pnorman> hstore+gin index?
18:07:27 <pnorman> partial hstore?
18:08:09 <zere> same spec, right? 4-8 cores, 24GB RAM, some decent but not necessarily amazingly fast disk?
18:08:09 <pnorman> last line was < pnorman> partial hstore?, did it all go through?
18:08:17 <zere> yup, i got all that.
18:09:15 <pnorman> decent disk as a minimum, SSD preferred as more people are going to them and it also better reflects servers handling extracts in-memory
18:11:26 <zere> how much, roughly?
18:12:32 <pnorman> space? checking numbers, but at least 300GB
18:14:35 <pnorman> <480GB
18:16:02 <zere> 500GB SSD... nice ;-)
18:16:10 <pnorman> 2x240
18:16:15 <apmon_p> the wikipeadia toolserver might give some indication of how quickly clustering degrades and what it achieves
18:16:50 <apmon_p> you do see a continous degredation of render performance
18:17:29 <apmon_p> although I'd need to run a new cluster to check if perfoance can be recovered
18:17:41 <apmon_p> but in the past I believe it did
18:17:56 <pnorman> frederik did his benchmarking on 3x120
18:21:35 <pnorman> to actually do the benchmarking I need to patch osm2pgsql to not cluster or create rendering indexes, so it's not the type of thing that if given a machine this morning I could start this instant. the two things I need help with are a) more things to benchmark, b) some idea of a chance of hardware
18:23:28 <zere> pnorman: i just ordered an EX40-SSD, and i'll let you know when it's ready ;-)
18:24:18 <iandees> pnorman zere: pnorman has been doing these great benchmarks
18:24:26 <iandees> we should think of somewhere for him to publish them
18:24:35 <pnorman> there's that too :)
18:24:49 <zere> yeah, the Journal of OpenStreetMap!
18:24:54 <iandees> i know you're looking for a place to talk about your results, too. we could try to find some funding to send you somewhere
18:24:57 <zere> oh, wait... that acronym is taken
18:25:04 <pnorman> JoO!
18:25:07 <iandees> Annals of OpenStreetMap
18:25:28 <TomH> JOSMFEWG
18:25:41 <pnorman> zere: damnit, now I need to delve into osm2pgsql code :)
18:26:13 <zere> pnorman: i've heard hetzner can take weeks to set these things up properly ;-)
18:26:25 <pnorman> hetzner has been really fast for me
18:27:13 <zere> i've heard some horror stories of them coming up missing drives, 50%+ packet loss, etc... we'll see. this is the first thing i've got from them
18:28:57 <zere> about the publishing thing - is it worth pushing it out as a blog post, or do you think it's too technical for the main blog? perhaps we need something like https://blog.twitter.com/engineering - but would there be a steady stream of content?
18:29:20 <RichardF> an @OSMTech account would be good.
18:29:38 <RichardF> Firefishy's own sysadmin tweets are always well received.
18:30:00 <RichardF> @OpenStreetMap is a bit hidebound for some audiences by having to be "accessible"
18:30:00 <ewg-meetbot`> RichardF: Error: "OpenStreetMap" is not a valid command.
18:30:05 <RichardF> oh fuck off ewg-meetbot`
18:30:54 <zere> damn. now i'm going to have to get my redaction sharpie out for the minutes :-(
18:31:13 <pnorman> you mean the impact of multiple osm2pgsql processes on way column correlation isn't accessible?
18:33:28 <apmon_p> osm2pgsql seems to be used a fair amount. so I think there would be a reasonable interest in such technical posts
18:33:59 <zere> yup. woodpeck's SOTM presentations on the subject were always well received.
18:34:14 <apmon_p> Frederick's talk on tuning osm2pgsql and mapnik seemed to get good resonances
18:34:20 <zere> perhaps it's something to write up and we can ask / lean on CWG as appropriate
18:34:40 <pnorman> I always wonder what to do with my results
18:35:00 <zere> good knowledge to have out there, and worth recognising pnorman as the pre-eminent expert on the subject ;-)
18:35:32 <apmon_p> we should probably have a page, e.g. on switch2osm where we collect those fact based best practices and results
18:37:22 <zere> awesome. hopefully hetzner will give us something to benchmark on soon.
18:37:29 <pnorman> but not too soon
18:37:37 <zere> was there anything else that anyone wanted to discuss (v. quickly)?
18:42:21 <pnorman> who's the contact for openstreetmap on github?
18:42:34 <iandees> isn't avar?
18:42:43 <TomH> define contact
18:42:56 <TomH> Both avar and I have admin acces
18:43:04 <zere> admin, for purposes of adding new projects?
18:43:05 <pnorman> I want to get openstreetmap-license-change on there
18:43:45 <TomH> I can create repos sure
18:46:06 <TomH> https://github.com/openstreetmap/openstreetmap-license-change
18:46:49 <pnorman> thanks. can you give me and matt owner access to the repo
18:48:46 <TomH> done
18:49:52 <pnorman> now I just need to understand ruby
18:52:25 <pnorman> and finish my resume and find a job. :(
19:01:00 <zere> oops, forgot to close the meeting... thanks everyone for coming!