org.ka2ddo.yaac.osm

Class OsmPbfSegmenter

java.lang.Object

org.ka2ddo.yaac.osm.OsmSegmenter

org.ka2ddo.yaac.osm.OsmPbfSegmenter

All Implemented Interfaces:

AbortListener, OsmSegmenter.ReportUpdates

public class OsmPbfSegmenter
extends OsmSegmenter
implements AbortListener, OsmSegmenter.ReportUpdates

This class parses an OpenStreetMaps PBF (protocol buffers) file into a series of one-degree "square" tiles of map data, stored in a compact binary format suitable for efficient and fast rendering by YAAC. This format is documented at https://wiki.openstreetmap.org/wiki/PBF_Format.

Special Java command-line parameters can be defined with the -D option to alter the operation of this class. These are:

• alt.tile.dir=/directory/path - specify where to put the temporary files that are used as a brute-force indexed OSM Node position table. Default is the configured tile directory specified in the YAAC Preferences, but a partition on another disk spindle could be used to reduce disk seeking latency.
• skip.osm.xml.parse - if defined, skip reading the OSM input file and just re-sort and merge any new *.ways.tmp and *.nodes.tmp files into the operational *.ways and *.nodes files. Useful if the import crashes prior to completing the sort and merge phase.
• dont.purge.temp - if defined, don't delete the temporary files when the processing is complete.

Author:

Andrew Pavlin, KA2DDO

Nested Class Summary

Nested classes/interfaces inherited from class org.ka2ddo.yaac.osm.OsmSegmenter

OsmSegmenter.AcceptTileSubdirsOrPlanetWays, OsmSegmenter.BoundsToNameTag, OsmSegmenter.OpenRandomAccessFileCache, OsmSegmenter.ReportUpdates, OsmSegmenter.WayFileSorter

Field Summary

Fields inherited from class org.ka2ddo.yaac.osm.OsmSegmenter

emptyPostalQueueIdle, ESTIMATED_EXTRA_TILE_SPACE, ESTIMATED_TEMP_TILE_SPACE, FILETYPE_SUFFIXES, FILETYPE_SWAY, FILETYPE_WAY, flushUntiled, fullPostalQueueDelay, listener, MAX_LATITUDE, MAX_LONGITUDE, MAX_WAY_READ_FILES, MIN_LATITUDE, MIN_LONGITUDE, minWaysToWrite, numAddressesSaved, numDupPostalQueueEmpty, numDupPostalQueueFull, numDupWaysDeleted, numInvalidPostal, numNodeFiles, numPatchedAddresses, numPostalQueueEmpty, numPostalQueueFull, numRelations, numSortFailures, numTileWays, numWayFiles, numWayReaderOpens, numWayReads, out, planetDos, RELATIONS_STATS_MAP_ENTRY_COMPARATOR, S_LOG_TIMESTAMP_FMT, sdfLogTimestamp, tileRootDirectory, tileTmpDirectory, unsavedWayFileCache, unsavedWaysFileCounter, untiledDos, WAY_INDEX_PREFILL, WAY_READBACK_BUF_SIZE, WAY_READBACK_MAX_BUFFERS, wayIdUsedSet, wayIndex, wayQueuesAreEmpty, wayWriterQueues, wayWriterThread, zeroLengthWayIds

Constructor Summary

Constructors

Constructor and Description
OsmPbfSegmenter(java.lang.String rootTilePath) Constructor for protocol buffers OSM file segmenter,

Method Summary

Modifier and Type	Method and Description
void	cancelRequested() Called from the user interface when the user requests a cancel.
protected long	getMinHeapRequired() Check if there is enough heap available for the importer to operate without guaranteed OutOfMemoryErrors due to the minimum size of data structures that will be simultaneously allocated during import.
void	parse(java.io.FileInputStream fileInputStream, NonshareableCountingBufferedDataInputStream inputStream, java.io.File inputFile, boolean useFixMe, StatusListener listener) Parse an OSM PBF file into YAAC's version of OpenStreetMap tiles.
void	updateMessage(java.lang.String countString) Report a status update string to the progress indicator (if one was provided).

Methods inherited from class org.ka2ddo.yaac.osm.OsmSegmenter

checkDiskSpace, closePostalAddresses, computeActiveQueueCount, computeQueuesBacklog, copyAddressFromNodeToOther, ensurePostalTablesExist, findBiggestQueue, flushPostalAddressesToDB, getFileForIndexedWay, getPostalAddressQueueSize, getWayForRelation, isPostalDbInUse, processNonStdAmenityTypes, produceSortStatusMessage, refreshMapWindows, tryToSetUpPostalDB, updateAddressesWithStreet, writePostalAddressToDb

Methods inherited from class java.lang.Object

clone, equals, finalize, getClass, hashCode, notify, notifyAll, toString, wait, wait, wait

Constructor Detail

OsmPbfSegmenter

public OsmPbfSegmenter(java.lang.String rootTilePath)
                throws java.io.IOException

Constructor for protocol buffers OSM file segmenter,

Parameters:

rootTilePath - String pathname of the configured tile directory

Throws:

java.io.IOException - if tile directory cannot be created

Method Detail

getMinHeapRequired

protected long getMinHeapRequired()

Check if there is enough heap available for the importer to operate without guaranteed OutOfMemoryErrors due to the minimum size of data structures that will be simultaneously allocated during import.

Specified by:

getMinHeapRequired in class OsmSegmenter

Returns:

localized error message String, or null if no problem detected

parse

public void parse(java.io.FileInputStream fileInputStream,
                  NonshareableCountingBufferedDataInputStream inputStream,
                  java.io.File inputFile,
                  boolean useFixMe,
                  StatusListener listener)
           throws java.io.IOException

Parse an OSM PBF file into YAAC's version of OpenStreetMap tiles.

Specified by:

parse in class OsmSegmenter

Parameters:

fileInputStream - FileInputStream underlying the input stream being used

inputStream - NonshareableCountingBufferedDataInputStream reading a PBF file

inputFile - File of the OpenStreetMap snapshot file being read

useFixMe - boolean true if FIXME-tagged elements of the OSM data are to be included

listener - StatusListener to receive progress report of the import

Throws:

java.io.IOException - if the input file cannot be read or any of the output files cannot be written

cancelRequested

public void cancelRequested()

Called from the user interface when the user requests a cancel. Implementations of this method should not block, but should quickly inform the associated long-running background thread to abort its operation.

Specified by:

cancelRequested in interface AbortListener

updateMessage

public void updateMessage(java.lang.String countString)
                   throws UserAbort

Report a status update string to the progress indicator (if one was provided).

Specified by:

updateMessage in interface OsmSegmenter.ReportUpdates

Parameters:

countString - String to report

Throws:

UserAbort - if user clicked the abort button