Package org.ka2ddo.yaac.osm

Class OsmSegmenter

java.lang.Object
org.ka2ddo.yaac.osm.OsmSegmenter
Direct Known Subclasses:
OsmPbfSegmenter, OsmXmlSegmenter
public abstract class OsmSegmenter extends Object
Common code for different OpenStreetMap dataset file format loaders.
Author:
Andrew Pavlin, KA2DDO

  • Field Details

    • ESTIMATED_EXTRA_TILE_SPACE

      public static final long ESTIMATED_EXTRA_TILE_SPACE
      Kilobytes of free disk space needed on the tile directory's partition to handle importing a new OSM file.
      See Also:

    • ESTIMATED_TEMP_TILE_SPACE

      public static final long ESTIMATED_TEMP_TILE_SPACE
      Kilobytes of free disk space needed on the temporary directory's partition to handle importing a new OSM file. The temporary directory is used instead of the tile directory when the Java system property alt.tile.dir is defined; if not defined, this usage is added to the tile directory's usage.
      See Also:

    • MAX_LATITUDE

      protected static final int MAX_LATITUDE
      Maximum latitude in millionths of degrees.
      See Also:

    • MIN_LATITUDE

      protected static final int MIN_LATITUDE
      Minimum latitude in millionths of degrees.
      See Also:

    • MAX_LONGITUDE

      protected static final int MAX_LONGITUDE
      Maximum longitude in millionths of degrees.
      See Also:

    • MIN_LONGITUDE

      protected static final int MIN_LONGITUDE
      Minimum longitude in millionths of degrees.
      See Also:

    • FILETYPE_SUFFIXES

      protected static final String[] FILETYPE_SUFFIXES
      Filetypes for different types of Way files.
      See Also:

    • FILETYPE_WAY

      protected static final int FILETYPE_WAY
      Flag indicating OSM Ways properly labeled for rendering.
      See Also:

    • FILETYPE_SWAY

      protected static final int FILETYPE_SWAY
      Flag indicating OSM Ways not properly labeled for rendering.
      See Also:

    • RELATIONS_STATS_MAP_ENTRY_COMPARATOR

      protected static final org.ka2ddo.yaac.osm.OsmSegmenter.RelationsStatsMapEntryComparator RELATIONS_STATS_MAP_ENTRY_COMPARATOR

    • tileRootDirectory

      protected static File tileRootDirectory
      File identifying the directory used by YAAC to store map tile file latitude directories.

    • tileTmpDirectory

      protected final File tileTmpDirectory
      File identifying the directory used by OpenStreetMap importers for high-data-access-rate temporary files. If not defined as a separate location, has the same value as tileRootDirectory.
      See Also:

    • planetDos

      protected transient NonshareableBufferedDataOutputStream planetDos
      Shared area for stream to planet.ways.tmp file. Used for ocean way processing.

    • numWayFiles

      protected transient int numWayFiles
      Counter of processed Way files (used by common reporting code).

    • numNodeFiles

      protected transient int numNodeFiles
      Counter of processed Node files (used by common reporting code).

    • numSortFailures

      protected transient int numSortFailures
      Counter of failed sorts of Way and Node files.

    • numDupWaysDeleted

      protected transient int numDupWaysDeleted
      Counter of number of duplicate Ways deleted from the collection of Way files.

  • Constructor Details

    • OsmSegmenter

      protected OsmSegmenter(String rootTilePath) throws IOException
      Abstract pre-constructor for any OSM file segmenter,
      Parameters:
      rootTilePath - String pathname of the configured tile directory
      Throws:
      IOException - if tile directory cannot be created

  • Method Details

    • parse

      public abstract void parse(FileInputStream fileInputStream, NonshareableCountingBufferedDataInputStream inputStream, File inputFile, boolean useFixMe, StatusListener listener) throws IOException, ParserConfigurationException, SAXException
      Parse a OpenStreetMap snapshot file into YAAC's version of OpenStreetMap tiles.
      Parameters:
      fileInputStream - FileInputStream underlying the input stream being used
      inputStream - NonshareableCountingBufferedDataInputStream reading the snapshot file of whatever format the implementing class supports
      inputFile - File of the OpenStreetMap snapshot file being read
      useFixMe - boolean true if FIXME-tagged elements of the OSM data are to be included
      listener - StatusListener to receive progress report of the import
      Throws:
      IOException - if the input file cannot be read or any of the output files cannot be written
      ParserConfigurationException - if the Java runtime's XML parser cannot handle its configuration parameters
      SAXException - if the XML parser cannot decode the file

    • getMinHeapRequired

      protected abstract long getMinHeapRequired()
      Report the minimum total size of data structures that will be simultaneously allocated during import.
      Returns:
      long byte count for estimated minimum heap

    • checkDiskSpace

      public String checkDiskSpace(long filesize)
      Check if sufficient disk space is available to import the OSM file.
      Parameters:
      filesize - long byte-count size of the OSM file to read (to estimate resulting tile size)
      Returns:
      error message if not enough disk space, or null if no apparent problem

    • findBiggestQueue

      protected static GenericTaggedQueue findBiggestQueue(GenericTaggedQueue[] writerQueues, int upperLimitQueueSize)
      Helper method to find the queue with the largest backlog in an array of queues.
      Parameters:
      writerQueues - array of GenericTaggedQueues
      upperLimitQueueSize - int threshold of largest queue capacity, so scan can stop as soon as the fullest possible queue is found
      Returns:
      fullest GenericTaggedQueue in array, or null if all queues are empty

    • computeQueuesBacklog

      protected static int computeQueuesBacklog(GenericTaggedQueue[] queues)
      Helper method to compute the current backlog count of queued objects. This is not synchronized, so the count may be an approximation if other threads are inserting to or draining the queues.
      Parameters:
      queues - array of GenericTaggedQueues
      Returns:
      count of objects in all the queues in the array

    • computeActiveQueueCount

      protected static int computeActiveQueueCount(GenericTaggedQueue[] queues)
      Helper method to compute the current count of non-empty queues. This is not synchronized, so the count may be an approximation if other threads are inserting to or draining the queues.
      Parameters:
      queues - array of GenericTaggedQueues
      Returns:
      count of non-empty queues in the array

    • produceSortStatusMessage

      protected String produceSortStatusMessage(File latDir)
      Generate a string reporting the current state of sorting the tile files.
      Parameters:
      latDir - File of the latitude diectory currently being processed
      Returns:
      formatted string of import progress

    • refreshMapWindows

      protected static void refreshMapWindows()
      Tell all map windows to redraw their maps based upon the updated map tile files.