Attendees
| IRC nick
|
Real name
|
| pnorman |
Paul Norman
|
| shaunmcdonald |
Shaun McDonald
|
| zere |
Matt Amos
|
Summary
- Hack day organisation / recommendations:
- EWG received an email from the organisers of the post-SOTM hackday with some very useful feedback.
- EWG will try to collate recommendations based on collective experience here [1] and publish it to help out future organisers.
- Time change / DST
- Various places are going through DST changes at the moment, and the EWG time is changing to 17:30 UTC.
- Benchmarking osm2pgsql
- pnorman is benchmarking the osm2pgsql threading branch, and was looking for a physical machine with 4-8 cores, 24GB RAM and otherwise idle.
IRC Log
17:01:10 <zere> minutes of the last meeting, please let me know if anything needs changing: http://www.osmfoundation.org/wiki/Working_Group_Minutes/EWG_2013-10-21
17:02:42 <zere> we had a very useful email from some of the organisers of the post-SOTM hackday
17:02:53 <pnorman> minutes agree with my recollection
17:03:11 <zere> areas where they would have liked some support might be summarised as:
17:04:01 <zere> networking - they weren't expecting to have issue with the wifi, but there were - possibly capacity, possibly number of devices.
17:04:20 <zere> communication - getting the word out about the hackday and attracting people
17:05:01 <zere> ideas for projects - the non-devs attending and organising had a hard time keeping track of what the current projects are.
17:06:22 <zere> post-event wrap up - communicating the results of the event, what was done, and putting it somewhere to attract people who weren't at the event, and attract more people to the next one.
17:06:57 <zere> and they weren't aware that funding was available. but thankfully it was high profile enough that they were able to get a sponsor.
17:07:02 <zere> so...
17:07:03 <pnorman> taking up the first one, the sheer device density will bog down the available frequencies. the best method I've seen for dealing with it is to have wired connections available for those who can use them because every person on a wired connection means more available wireless bandwidth, and the wired is nearly always faster
17:07:18 <shaunmcdonald> On the networking we were told that it was going to work fine for the 30 people that we had registered, but then 60 people turned up (which is what we had the room booked for).
17:07:32 <shaunmcdonald> Other people in the building were having problems too.
17:08:15 <zere> i guess in terms of raw capacity, there's not much we can do except advise people looking for venues to ask after the capacity of their uplink?
17:08:15 <shaunmcdonald> Unfortunately they only had 1 or 2 wired sockets in the room.
17:08:40 <pnorman> I bring a cheap gigabit switch to events like that
17:08:48 <zere> as for wifi - does it work to bring some extra wifi routers, and re-export the wifi connection on other channels?
17:09:01 <zere> or re-export it as wired?
17:09:33 <shaunmcdonald> The other problem, is that people arrive with multiple devices and use a lot more data than the average conference. Having a local cache of certain files would be useful.
17:10:11 <zere> i'm assuming if the issue is high numbers of devices, then moving devices to a wired network, or other wireless network, would improve the situation. but is that true?
17:10:38 <pnorman> yes it is
17:10:40 <shaunmcdonald> Taking another wifi router can sometimes work, esp if it can use the 5GHz spectrum, which some places don't use so much yet and is less congested.
17:11:25 <zere> and yes, the "having a local cache" came up in a previous meeting. do you think it's worth compiling a list of stuff, say on a USB drive, which could be passed around?
17:11:55 <pnorman> I know at day 1 of sotm-us I used probably 2-3 GB of bandwidth on software installs so not sure how much use it is
17:12:07 <zere> or a more technological solution, if the wifi is being interposed anyway, that it would all be transparently proxied?
17:12:28 <zere> of course, transparent proxy is worthless for github ssh/https clones...
17:13:58 <shaunmcdonald> part of the problem is software being installed over the web.
17:16:10 <pnorman> I'd suggest we recommend a) maximizing use of wired connections, including bringing switches if necessary, b) not bringing stuff on USB for shared resources because it's unlikely anyone is hacking on a full planet on a laptop, c) recommending git clone --depth=1 in limited-bandwidth situations, d) recommending people pre-install software they know they'll need
17:16:12 <zere> from apt / yum?
17:18:18 <pnorman> e) making sure the building owners are aware that it's a hack day, not a conference, and everyone will have and be using 1+ wireless devices working primarily online, f) making sure there are accurate esimates of people given to the owner
17:19:00 <shaunmcdonald> e) make that 2+ wireless devices.
17:19:35 <pnorman> e) ..., and everyone will have 2+ wireless devices and actively be using 1+
17:20:31 <zere> sadly (f) is always going to be a difficult one. requiring early sign-up limits opportunistic attendance, any many people only make their mind up to attend in a few days immediately preceding the event.
17:21:41 <pnorman> f) estimates of people given to the owner, including the percentage of additional people who are likely to show up on the day of the event
17:22:14 <zere> yup. perhaps that's something we can help with - figuring out what a "normal" level of additional people is...
17:22:30 <pnorman> let me type these up a bit nicer
17:22:50 <zere> worth knowing that for food, also.
17:23:31 <shaunmcdonald> One problem that we had was people signing up in the last day or two after the numbers had been sent, plus an announcement at the end of the conference which got more people coming along.
17:23:53 <zere> at the moment, i've got 5 people signed up for http://wiki.openstreetmap.org/wiki/London/London_Hack_Weekend_Nov_2013
17:24:02 <zere> i'm hoping that a bit more than 5 will turn up ;-)
17:24:35 <shaunmcdonald> Yeah there was issues in the Post SOTM hack day with food too.
17:24:46 <zere> variety? amount?
17:25:21 <zere> i usually go by rule of thumb that it's 2/3 pizza per person. we're a hungry bunch, and it's always better to have too much food than too little
17:25:34 <shaunmcdonald> amount, plus online pizza orders having a max of £100. And the venue putting us in the wrong day in the diary.
17:25:54 <shaunmcdonald> what size?
17:26:19 <pnorman> https://hackpad.com/EWG-hack-day-recommendations-lh95anw01vN (still typing away)
17:28:26 <zere> size of pizza? hmmm... says 11" here
17:31:51 <zere> so, re: 1) "maximise the use of wired connections" - we would recommend bringing a switch, but what if there's no wired connection to plug into? worth also bringing in some sort of bridge, or a laptop to act as one?
17:33:18 <pnorman> Never tried either of those options so not sure
17:33:56 <shaunmcdonald> I have used the internet sharing option on my mac before which did the trick.
17:34:30 <zere> i'm thinking primarily of here... the wired network is private and locked down. i will see if i can get a port on the wired, external network.
17:35:44 <pnorman> using a bridge cuts the number of devices but not bandwidth, so it'd be of limited help.
17:35:54 <zere> yup
17:36:12 * pnorman got 500 KB/s on wired on SOTM-US day 1 while wireless were getting <100 KB/s
17:36:35 <zere> what were you downloading?
17:36:49 <shaunmcdonald> ah for organisations that don't allow clients to connect via a cable.
17:37:19 <pnorman> I was trying to get tilemill working
17:37:21 <shaunmcdonald> a caching proxy server that could be put in, may do the trick on the bandwidth.
17:37:26 <zere> yup, has to be MAC registered and so forth. IDSses light up like xmas trees when Firefishy plugs his laptop in ;-)
17:40:45 <pnorman> +100% on SOTM attendence over registration, right?
17:41:15 <zere> yeah, i think they expected 30, got 60.
17:41:34 <shaunmcdonald> that's right
17:41:49 <pnorman> do we have numbers for other days not attached to an event?
17:42:19 <zere> is it worth making some recommendations about "when all else fails"? perhaps bringing a 3G dongle, or making sure there's whiteboard + pens available?
17:42:33 <pnorman> I'd recommend whiteboard+pens always
17:42:49 * pnorman is a bit of a traditionalist
17:43:40 <pnorman> I know SOTM-US day 1 used them, are they commonly used elsewhere?
17:43:41 <zere> projector + screen?
17:43:58 <zere> we have some here, but i don't recall them being used a lot
17:45:04 <pnorman> the vancouver open data event used projector+screen but that one was a lot less focused and had wrap-up presentations from the groups who would probably never see each other again
17:45:12 <zere> iandees: ping? it looks like you had 12 signups for http://wiki.openstreetmap.org/wiki/Chicago_Hack_Weekend_April_2013 - do you remember how many came?
17:45:13 <shaunmcdonald> project and screen were used at the SotM Scotland hack day.
17:45:30 <shaunmcdonald> There's been a few others where they have been used.
17:46:13 <shaunmcdonald> Some people at the SotM hack day were using 3G, however I was struggling on that, thus is very provider dependant.
17:47:12 <zere> yeah, i'm just wondering what happens when the number of people present is sufficient to overload the wifi and bring it down / crawling permanently
17:53:20 <zere> i'm not entirely sure, but at the last event i had it looks like 15 people were signed up a week before the event, up to 22 the night before the event, and somewhere around 24-30 people turned up (but i didn't keep particularly good track of how many).
17:54:05 <zere> if 1 week before is an appropriate cut-off for telling venues, etc... how many people, then 2x is definitely not a disproporionate estimate.
17:55:47 <pnorman> food is a lot more last minute if you're going for delivered pizza
17:55:57 <zere> true
17:56:43 <shaunmcdonald> The venue at SotM was a bit confusing, as we had to use the on site catering, but then they were happy enough to let us get pizza delivered.
17:58:28 <pnorman> Does that look good for what we can recommend based on our experience? Might want to re-order it somehow, but that's all I can think of
17:59:22 <pnorman> I know some of that stuff is considered basic event organizating where it's the responsibility of the organizers, but we're hackers, not event organizers, so sometimes a reminder is good
17:59:30 <zere> i was thinking maybe we need some code-of-conduct style things too
18:00:03 <zere> in particular, we've had issues in the past around people taking pictures / people not wanting to be photographed.
18:01:40 <zere> we can have a think about it, and add anything we think of to that hackpad and revisit next week?
18:01:56 <pnorman> I'd keep this seperate from a code-of-conduct
18:02:19 <zere> speaking of: next week the US (and presumably canada) will go off DST. do we want to move the meeting up an hour, or keep it at 1700 UTC?
18:02:20 <shaunmcdonald> I'm sure event organisers have check lists to make sure that everything is included and the obvious isn't forgotten.
18:02:59 <zere> well... the issue there is that many of these hack events are organised by hackers. people like me who have no clue what's obvious to a proper event organiser.
18:03:18 <shaunmcdonald> zere: I prefer 1700 UTC.
18:03:27 * pnorman is not a morning person
18:04:42 <shaunmcdonald> Alternatively could go half way for 1730.
18:07:03 <zere> i don't mind half way. worth trying anything once ;-)
18:07:07 <zere> pnorman: how's that for you?
18:07:13 <pnorman> I can manage it
18:07:22 <pnorman> One last item, I'm getting some weird results with benchmarking osm2pgsql threading branch (e.g. indexing time depends on number of processes used in the previous parts of the import) and want to run some benchmarks on real hardware, not VMs. Anyone have resources I could use?
18:07:40 <zere> cool. we can always change the time again later if it becomes unmanagable for anyone.
18:08:07 <zere> also, shaunmcdonald suggested i ping reminders to people on this channel - if you want a reminder, let me know and i'll try and remember
18:08:18 <zere> like 30-60 mintues before the meetnig.
18:08:39 <zere> pnorman: what kind of spec do you need?
18:10:01 <pnorman> 4-8 threads (multithreading branch, so obvious needs) and if working with a full planet, 24 GB RAM. I can adjust size to disk space, but fast is good because it gets it CPU bound
18:11:31 * pnorman is wondering if the number of processes is impacting the ordering, resulting in differing sort times for the ORDER BY (psudo-cluster)
18:12:04 <pnorman> I mainly wonder that because I can't imagine any other possible reason.
18:13:20 <shaunmcdonald> I wonder is some company such as Bytemark or Hetzner would sponsor a server for you? Bytemark in the past have said they'd like to give more resources to OSM.
18:14:13 <pnorman> it's *hard* getting good performance benchmarks for the toolchain because there is *so* much stuff to tweak
18:14:47 <pnorman> and you need a large extract at the minimum to get real-world numbers
18:14:57 <zere> shaunmcdonald: yeah, i had an enquiry from them about hardware needs. i sent them a wish-list but i haven't heard back yet.
18:15:54 <zere> pnorman: http://www.hetzner.de/en/hosting/produkte_rootserver/ex6s ?
18:16:04 <pnorman> I'm saying this without double-checking 2x240GB in RAID0 is enough for the parallel CLUSTER, but I think hetzner EX40-SSD is enough
18:16:45 <pnorman> zere: rather the EX40-SSD, faster drives, same RAM. EX6S has ECC + Xeon features, but don't need those
18:17:19 <zere> and 3TB of disk space, so no worrying about space ;-)
18:17:31 <pnorman> ya but it's on 7200 RPM drives
18:18:06 <zere> i don't think we need the meeting open for this.... but thanks all for coming, and see you next week :-)