Osm2pgsql project 2020-07/Report

This report on the Osm2pgsql project grant was initially published on Jochen Topf's blog as “Osm2pgsql Work” on Wed 2020-12-02 17:27, and is archived here with permission.

Osm2pgsql Work

Back in September we launched the new osm2pgsql.org website. At that time it was still quite rudimentary. Over the last months a big part of my work for the OSM Foundation was to work on that site.

Osm2pgsql has been around for a long time but never had a real project web site. Most information about the project was in its git repository, some on the OSM wiki, and more information strewn here or there in blog posts etc. So my first job was to see what’s out there and what’s missing and create a structure that I could fill with content. Over time the website grew to contain all the sections that software projects tend to have, like documentation, installation instructions, news, support information, etc. Where possible I tried to use existing documentation, but it all had to be checked first, a lot of it was outdated or too complicated or slightly wrong. I filled it into the structure, re-wrote, re-organized, added more content, more documentation, examples. And I also removed much of the old content on the wiki pages or in the repository.

We also needed a bit of color, a few icons, a layout that works on big screens and small phones and in a screen reader. Over time the content and its presentation grew together. I am pretty happy with what we have now. Not perfect, not complete, not done, and it never will be. But I believe we have something now that will help new and old users alike to use osm2pgsql in their projects. And now that everything is in one place, it is much easier for the developers to fix something small here or there or add the documentation for new features that they are building. From now on the website can just be updated as part of the other work we are doing on osm2pgsql.

Two sections of the website I want to talk about in particular: The manual and the examples. Having a manual is, of course, necessary for any software. One document that, as a whole, should describe everything there is to know about osm2pgsql. There are some chapters that are still rather short, but there is lot written down now that was never before explained anywhere. But a manual only shows you the “how”, it’s great once you have decided that you want to use osm2pgql and want to figure out how to solve your problem with it. But often the question is: Can this tool even solve my problem? And that’s where the examples come in. They are not tutorials, but they show a wide range of problems and that and how osm2pgsql can solve them. I hope this gives current and potential users of osm2pgsql more of an idea what’s possible and where to start.

Making osm2pgsql more user friendly is not only about the website and documentation. Another part of that, also paid for by the OSMF, was going through the code and improving the logging output, error messages and such. All messages that the program produces have a time stamp now, the verbosity is configurable from “show only errors” to “show every SQL command and all the data sent to the database” for debugging. What’s shown is more concise and more consistent then before. And if you want to send the output to a log file, you can switch off the progress reports. All of these were long-standing open issues that I could close now.

I believe the result is much better than what we had before, but there were a lot of changes, so I am sure there are some place where I screwed up and some important message isn’t shown any more or so. But we’ll fix the problems as they come up.

Next up: We have some loose ends to tie up, but will publish a release soon. This contains not only all the changes I wrote about above but also many fixes and new features. But that’s for another blog post.