Package org.ka2ddo.util

Class NonshareableDoubleBufferedDataInputStream

java.lang.Object
java.io.InputStream
java.io.FilterInputStream
org.ka2ddo.util.NonshareableDoubleBufferedDataInputStream
All Implemented Interfaces:
Closeable, DataInput, AutoCloseable, Runnable
public final class NonshareableDoubleBufferedDataInputStream extends FilterInputStream implements DataInput, Runnable
This class is (sort of) a merged copy of java.io.BufferedInputStream and java.io.DataInputStream, but without the synchronization, which restricts instances of this class to use in a single thread. Note that, because of certain optimizations performed in this code, the user should either use a buffer size greater than 65536 or ensure that all strings read with the readUTF() method are shorter than the buffer size. This must be used on a dedicated FileInputStream, as it uses a separate Thread from an Executor pool to read into the first buffer so the second buffer can be drained simultaneously.
Author:
Andrew Pavlin, KA2DDO

  • Nested Class Summary

    Nested Classes
    Modifier and Type
    Class
    Description
    static interface 
    NonshareableDoubleBufferedDataInputStream.FileClosedListener
    This interface allows a handler to be informed when end-of-file is detected by the file-reading thread (typically so the next file can be opened and first buffer loaded before the data consumer finishes consuming this stream).

  • Field Summary

    Fields inherited from class java.io.FilterInputStream

    in

  • Constructor Summary

    Constructors
    Constructor
    Description
    NonshareableDoubleBufferedDataInputStream(InputStream in)
    Creates a NonshareableDoubleBufferedDataInputStream and saves its argument, the input stream in, for later use.
    NonshareableDoubleBufferedDataInputStream(InputStream in, byte[] buf1, byte[] buf2)
    Creates a NonshareableDoubleBufferedDataInputStream with the specified buffer size, and saves its argument, the input stream in, for later use.
    NonshareableDoubleBufferedDataInputStream(InputStream in, byte[] buf1, byte[] buf2, NonshareableDoubleBufferedDataInputStream.FileClosedListener listener, Object context)
    Creates a NonshareableDoubleBufferedDataInputStream with the specified buffer size, and saves its argument, the input stream in, for later use.
    NonshareableDoubleBufferedDataInputStream(InputStream in, int size)
    Creates a NonshareableDoubleBufferedDataInputStream with the specified buffer size, and saves its argument, the input stream in, for later use.

  • Method Summary

    Modifier and Type
    Method
    Description
    int
    available()
    Returns an estimate of the number of bytes that can be read (or skipped over) from this input stream without blocking by the next invocation of a method for this input stream.
    void
    close()
    Closes this input stream and releases any system resources associated with the stream.
    long
    getByteCount()
    Get the total number of bytes read through this stream (so far).
    int
    getNumBufNotReady()
    Report the number of times the file-reading thread had not provided sufficient data for the data-consuming thread when it wanted more data.
    int
    getSeqNum()
    Get the identifying sequence number of this instance of NonshareableDoubleBufferedDataInputStream.
    long
    getTimeWaitingForBuf()
    Return the total amount of time (in milliseconds) that the data-consuming thread had to wait because the file-reading thread wasn't fast enough.
    boolean
    isEof()
    Test if this stream's file-reading thread has reached end-of-file of the underlying stream.
    void
    mark(int readlimit)
    See the general contract of the mark method of InputStream.
    boolean
    markSupported()
    Tests if this input stream supports the mark and reset methods.
    int
    read()
    See the general contract of the read method of InputStream.
    final int
    read(byte[] b)
    Reads some number of bytes from the contained input stream and stores them into the buffer array b.
    int
    read(byte[] b, int off, int len)
    Reads bytes from this byte-input stream into the specified byte array, starting at the given offset.
    final boolean
    readBoolean()
    See the general contract of the readBoolean method of DataInput.
    final byte
    readByte()
    See the general contract of the readByte method of DataInput.
    final char
    readChar()
    See the general contract of the readChar method of DataInput.
    final double
    readDouble()
    See the general contract of the readDouble method of DataInput.
    final float
    readFloat()
    See the general contract of the readFloat method of DataInput.
    final void
    readFully(byte[] b)
    See the general contract of the readFully method of DataInput.
    final void
    readFully(byte[] b, int off, int len)
    See the general contract of the readFully method of DataInput.
    final int
    readInt()
    See the general contract of the readInt method of DataInput.
    final String
    readLine()
    See the general contract of the readLine method of DataInput.
    final int
    readLittleEndianInt()
    See the general contract of the readInt method of DataInput, except this reads the bytes in little-endian order rather than big-endian (network) byte order.
    final short
    readLittleEndianShort()
    See the general contract of the readShort method of DataInput.
    final long
    readLong()
    See the general contract of the readLong method of DataInput.
    final short
    readShort()
    See the general contract of the readShort method of DataInput.
    final int
    readUnsignedByte()
    See the general contract of the readUnsignedByte method of DataInput.
    final int
    readUnsignedShort()
    See the general contract of the readUnsignedShort method of DataInput.
    final String
    readUTF()
    See the general contract of the readUTF method of DataInput.
    final String
    readUTF8Line()
    This method reads a line of text encoded in UTF-8, terminated by a line ending (CR, LF, or CRLF) or EOF.
    final char
    readUTFChar()
    This method reads one UTF-8 encoded character from the input stream, using as many bytes as necessary to get an entire character.
    void
    reset()
    See the general contract of the reset method of InputStream.
    void
    run()
    DO NOT CALL.
    void
    setFileClosedListener(NonshareableDoubleBufferedDataInputStream.FileClosedListener l, Object context)
    Register (or update the registered listener if called more than once) to be informed when this file is closed.
    long
    skip(long n)
    See the general contract of the skip method of InputStream.
    final int
    skipBytes(int n)
    See the general contract of the skipBytes method of DataInput.

    Methods inherited from class java.io.InputStream

    nullInputStream, readAllBytes, readNBytes, readNBytes, skipNBytes, transferTo

    Methods inherited from class java.lang.Object

    clone, equals, finalize, getClass, hashCode, notify, notifyAll, toString, wait, wait, wait

  • Constructor Details

    • NonshareableDoubleBufferedDataInputStream

      public NonshareableDoubleBufferedDataInputStream(InputStream in)
      Creates a NonshareableDoubleBufferedDataInputStream and saves its argument, the input stream in, for later use. An internal buffer array is created and stored in buf.
      Parameters:
      in - the underlying input stream.

    • NonshareableDoubleBufferedDataInputStream

      public NonshareableDoubleBufferedDataInputStream(InputStream in, int size)
      Creates a NonshareableDoubleBufferedDataInputStream with the specified buffer size, and saves its argument, the input stream in, for later use. An internal buffer array of length size is created and stored in buf.
      Parameters:
      in - the underlying input stream.
      size - the buffer size (minimum size 8 bytes).
      Throws:
      IllegalArgumentException - if size < 8.

    • NonshareableDoubleBufferedDataInputStream

      public NonshareableDoubleBufferedDataInputStream(InputStream in, byte[] buf1, byte[] buf2)
      Creates a NonshareableDoubleBufferedDataInputStream with the specified buffer size, and saves its argument, the input stream in, for later use. The specified pair of buffers (which must be the same length) are used for file-reading and data extraction.
      Parameters:
      in - the underlying input stream.
      buf1 - the first ByteBuffer to use for the double-buffered reader
      buf2 - the second ByteBuffer to use for the double-buffered reader; must be same capacity as the first one
      Throws:
      IllegalArgumentException - if size < 8.

    • NonshareableDoubleBufferedDataInputStream

      public NonshareableDoubleBufferedDataInputStream(InputStream in, byte[] buf1, byte[] buf2, NonshareableDoubleBufferedDataInputStream.FileClosedListener listener, Object context)
      Creates a NonshareableDoubleBufferedDataInputStream with the specified buffer size, and saves its argument, the input stream in, for later use. The specified pair of buffers (which must be the same length) are used for file-reading and data extraction, and registers the callback listener immediately (useful if the file being read is smaller than the buffer size, such that the callback might be called before the constructed object can be called in the thread that called this constructor).
      Parameters:
      in - the underlying input stream.
      buf1 - the first ByteBuffer to use for the double-buffered reader
      buf2 - the second ByteBuffer to use for the double-buffered reader; must be same capacity as the first one
      listener - FileClosedListener to be notified when file Eof is reached
      context - Object to be passed to listener (if there is a listener)
      Throws:
      IllegalArgumentException - if size < 8.

  • Method Details

    • setFileClosedListener

      public void setFileClosedListener(NonshareableDoubleBufferedDataInputStream.FileClosedListener l, Object context)
      Register (or update the registered listener if called more than once) to be informed when this file is closed.
      Parameters:
      l - FileClosedListener to be notified when file Eof is reached
      context - Object to be passed to listener (if there is a listener)

    • run

      public void run()
      DO NOT CALL. Reserved for background buffer filling. This is running (or in a wait state) the entire time the input file is open; the thread is only returned to the executor pool when the file is closed, so the pool needs at least two threads per simultaneously used chains of NonshareableDoubleBufferedDataInputStreams, so that the next stream can get a thread while this stream's thread is still cleaning up.
      Specified by:
      run in interface Runnable

    • read

      public int read() throws IOException
      See the general contract of the read method of InputStream.
      Overrides:
      read in class FilterInputStream
      Returns:
      the next byte of data, or -1 if the end of the stream is reached.
      Throws:
      IOException - if this input stream has been closed by invoking its close() method, or an I/O error occurs.

    • read

      public int read(byte[] b, int off, int len) throws IOException
      Reads bytes from this byte-input stream into the specified byte array, starting at the given offset.

      This method implements the general contract of the corresponding read method of the InputStream class. As an additional convenience, it attempts to read as many bytes as possible by repeatedly invoking the read method of the underlying stream. This iterated read continues until one of the following conditions becomes true:

      • The specified number of bytes have been read,
      • The read method of the underlying stream returns -1, indicating end-of-file, or
      • The available method of the underlying stream returns zero, indicating that further input requests would block.
      If the first read on the underlying stream returns -1 to indicate end-of-file then this method returns -1. Otherwise this method returns the number of bytes actually read.

      Subclasses of this class are encouraged, but not required, to attempt to read as many bytes as possible in the same fashion.

      Overrides:
      read in class FilterInputStream
      Parameters:
      b - destination buffer.
      off - offset at which to start storing bytes.
      len - maximum number of bytes to read.
      Returns:
      the number of bytes read, or -1 if the end of the stream has been reached.
      Throws:
      IOException - if this input stream has been closed by invoking its close() method, or an I/O error occurs.

    • skip

      public long skip(long n) throws IOException
      See the general contract of the skip method of InputStream.
      Overrides:
      skip in class FilterInputStream
      Throws:
      IOException - if the stream does not support seek, or if this input stream has been closed by invoking its close() method, or an I/O error occurs.

    • available

      public int available() throws IOException
      Returns an estimate of the number of bytes that can be read (or skipped over) from this input stream without blocking by the next invocation of a method for this input stream. The next invocation might be the same thread or another thread. A single read or skip of this many bytes will not block, but may read or skip fewer bytes.

      This method returns the number of bytes remaining to be read in the buffer (count - pos), possibly refilling the buffer if it was empty at the time of this call. Specifically, it does _not_ try to return the total remaining bytes in the file.

      Overrides:
      available in class FilterInputStream
      Returns:
      an estimate of the number of bytes that can be read (or skipped over) from this input stream without blocking.
      Throws:
      IOException - if this input stream has been closed by invoking its close() method, or an I/O error occurs.

    • mark

      public void mark(int readlimit)
      See the general contract of the mark method of InputStream.
      Overrides:
      mark in class FilterInputStream
      Parameters:
      readlimit - the maximum limit of bytes that can be read before the mark position becomes invalid.
      See Also:

    • reset

      public void reset() throws IOException
      See the general contract of the reset method of InputStream.

      If markpos is -1 (no mark has been set or the mark has been invalidated), an IOException is thrown. Otherwise, pos is set equal to markpos.

      Overrides:
      reset in class FilterInputStream
      Throws:
      IOException - if this stream has not been marked or, if the mark has been invalidated, or the stream has been closed by invoking its close() method, or an I/O error occurs.
      See Also:

    • markSupported

      public boolean markSupported()
      Tests if this input stream supports the mark and reset methods. The markSupported method of NonshareableDoubleBufferedDataInputStream returns false.
      Overrides:
      markSupported in class FilterInputStream
      Returns:
      a boolean indicating if this stream type supports the mark and reset methods.
      See Also:

    • close

      public void close() throws IOException
      Closes this input stream and releases any system resources associated with the stream. Once the stream has been closed, further read(), available(), reset(), or skip() invocations will throw an IOException; however, this class does allow you to read unread data that was buffered prior to the close() call. Closing a previously closed stream has no effect.
      Specified by:
      close in interface AutoCloseable
      Specified by:
      close in interface Closeable
      Overrides:
      close in class FilterInputStream
      Throws:
      IOException - if an I/O error occurs.

    • read

      public final int read(byte[] b) throws IOException
      Reads some number of bytes from the contained input stream and stores them into the buffer array b. The number of bytes actually read is returned as an integer. This method blocks until input data is available, end of file is detected, or an exception is thrown.

      If b is null, a NullPointerException is thrown. If the length of b is zero, then no bytes are read and 0 is returned; otherwise, there is an attempt to read at least one byte. If no byte is available because the stream is at end of file, the value -1 is returned; otherwise, at least one byte is read and stored into b.

      The first byte read is stored into element b[0], the next one into b[1], and so on. The number of bytes read is, at most, equal to the length of b. Let k be the number of bytes actually read; these bytes will be stored in elements b[0] through b[k-1], leaving elements b[k] through b[b.length-1] unaffected.

      The read(b) method has the same effect as: 

       read(b, 0, b.length)
      Overrides:
      read in class FilterInputStream
      Parameters:
      b - the buffer into which the data is read.
      Returns:
      the total number of bytes read into the buffer, or -1 if there is no more data because the end of the stream has been reached.
      Throws:
      IOException - if the first byte cannot be read for any reason other than end of file, the stream has been closed and the underlying input stream does not support reading after close, or another I/O error occurs.
      See Also:

    • readFully

      public final void readFully(byte[] b) throws IOException
      See the general contract of the readFully method of DataInput.
      Specified by:
      readFully in interface DataInput
      Parameters:
      b - the buffer into which the data is read.
      Throws:
      EOFException - if this input stream reaches the end before reading all the bytes.
      IOException - the stream has been closed and the contained input stream does not support reading after close, or another I/O error occurs.

    • readFully

      public final void readFully(byte[] b, int off, int len) throws IOException
      See the general contract of the readFully method of DataInput.
      Specified by:
      readFully in interface DataInput
      Parameters:
      b - the buffer into which the data is read.
      off - the start offset of the data.
      len - the number of bytes to read.
      Throws:
      EOFException - if this input stream reaches the end before reading all the bytes.
      IOException - the stream has been closed and the contained input stream does not support reading after close, or another I/O error occurs.

    • skipBytes

      public final int skipBytes(int n) throws IOException
      See the general contract of the skipBytes method of DataInput.
      Specified by:
      skipBytes in interface DataInput
      Parameters:
      n - the number of bytes to be skipped.
      Returns:
      the actual number of bytes skipped.
      Throws:
      IOException - if the contained input stream does not support seek, or the stream has been closed and the contained input stream does not support reading after close, or another I/O error occurs.

    • readBoolean

      public final boolean readBoolean() throws IOException
      See the general contract of the readBoolean method of DataInput.
      Specified by:
      readBoolean in interface DataInput
      Returns:
      the boolean value read.
      Throws:
      EOFException - if this input stream has reached the end.
      IOException - the stream has been closed and the contained input stream does not support reading after close, or another I/O error occurs.

    • readByte

      public final byte readByte() throws IOException
      See the general contract of the readByte method of DataInput.
      Specified by:
      readByte in interface DataInput
      Returns:
      the next byte of this input stream as a signed 8-bit byte.
      Throws:
      EOFException - if this input stream has reached the end.
      IOException - the stream has been closed and the contained input stream does not support reading after close, or another I/O error occurs.

    • readUnsignedByte

      public final int readUnsignedByte() throws IOException
      See the general contract of the readUnsignedByte method of DataInput.
      Specified by:
      readUnsignedByte in interface DataInput
      Returns:
      the next byte of this input stream, interpreted as an unsigned 8-bit number.
      Throws:
      EOFException - if this input stream has reached the end.
      IOException - the stream has been closed and the contained input stream does not support reading after close, or another I/O error occurs.

    • readShort

      public final short readShort() throws IOException
      See the general contract of the readShort method of DataInput.
      Specified by:
      readShort in interface DataInput
      Returns:
      the next two bytes of this input stream, interpreted as a signed 16-bit number.
      Throws:
      EOFException - if this input stream reaches the end before reading two bytes.
      IOException - the stream has been closed and the contained input stream does not support reading after close, or another I/O error occurs.

    • readUnsignedShort

      public final int readUnsignedShort() throws IOException
      See the general contract of the readUnsignedShort method of DataInput.
      Specified by:
      readUnsignedShort in interface DataInput
      Returns:
      the next two bytes of this input stream, interpreted as an unsigned 16-bit integer.
      Throws:
      EOFException - if this input stream reaches the end before reading two bytes.
      IOException - the stream has been closed and the contained input stream does not support reading after close, or another I/O error occurs.

    • readLittleEndianShort

      public final short readLittleEndianShort() throws IOException
      See the general contract of the readShort method of DataInput.
      Returns:
      the next two bytes of this input stream, interpreted as a signed 16-bit number.
      Throws:
      EOFException - if this input stream reaches the end before reading two bytes.
      IOException - the stream has been closed and the contained input stream does not support reading after close, or another I/O error occurs.

    • readChar

      public final char readChar() throws IOException
      See the general contract of the readChar method of DataInput.
      Specified by:
      readChar in interface DataInput
      Returns:
      the next two bytes of this input stream, interpreted as a char.
      Throws:
      EOFException - if this input stream reaches the end before reading two bytes.
      IOException - the stream has been closed and the contained input stream does not support reading after close, or another I/O error occurs.

    • readUTFChar

      public final char readUTFChar() throws EOFException, UTFDataFormatException, IOException
      This method reads one UTF-8 encoded character from the input stream, using as many bytes as necessary to get an entire character. Note that this method cannot support Unicode characters represented by the UTF-16 high and low surrogate sequence as decoded from a 4-byte UTF-8 character, as it would require returning 2 Java chars and this method can only return one.
      Returns:
      the Unicode character corresponding to the one to four bytes read from the input stream.
      Throws:
      EOFException - if this input stream reaches the end before reading an entire UTF-8 character.
      UTFDataFormatException - if the first few bytes in the stream do not represent a valid modified UTF-8 encoding of a Unicode character or the stream ends before the completion of a multi-byte UTF-8 character.
      IOException - the stream has been closed and the contained input stream does not support reading after close, or another I/O error occurs.

    • readInt

      public final int readInt() throws IOException
      See the general contract of the readInt method of DataInput.
      Specified by:
      readInt in interface DataInput
      Returns:
      the next four bytes of this input stream, interpreted as an int.
      Throws:
      EOFException - if this input stream reaches the end before reading four bytes.
      IOException - the stream has been closed and the contained input stream does not support reading after close, or another I/O error occurs.

    • readLittleEndianInt

      public final int readLittleEndianInt() throws IOException
      See the general contract of the readInt method of DataInput, except this reads the bytes in little-endian order rather than big-endian (network) byte order.
      Returns:
      the next four bytes of this input stream, interpreted as an int.
      Throws:
      EOFException - if this input stream reaches the end before reading four bytes.
      IOException - the stream has been closed and the contained input stream does not support reading after close, or another I/O error occurs.

    • readLong

      public final long readLong() throws IOException
      See the general contract of the readLong method of DataInput.
      Specified by:
      readLong in interface DataInput
      Returns:
      the next eight bytes of this input stream, interpreted as a long.
      Throws:
      EOFException - if this input stream reaches the end before reading eight bytes.
      IOException - the stream has been closed and the contained input stream does not support reading after close, or another I/O error occurs.

    • readFloat

      public final float readFloat() throws IOException
      See the general contract of the readFloat method of DataInput.
      Specified by:
      readFloat in interface DataInput
      Returns:
      the next four bytes of this input stream, interpreted as a float.
      Throws:
      EOFException - if this input stream reaches the end before reading four bytes.
      IOException - the stream has been closed and the contained input stream does not support reading after close, or another I/O error occurs.
      See Also:

    • readDouble

      public final double readDouble() throws IOException
      See the general contract of the readDouble method of DataInput.

      Bytes for this operation are read from the contained input stream.

      Specified by:
      readDouble in interface DataInput
      Returns:
      the next eight bytes of this input stream, interpreted as a double.
      Throws:
      EOFException - if this input stream reaches the end before reading eight bytes.
      IOException - the stream has been closed and the contained input stream does not support reading after close, or another I/O error occurs.
      See Also:

    • readLine

      public final String readLine() throws IOException
      See the general contract of the readLine method of DataInput.

      Bytes for this operation are read from the contained input stream. Note that this method only supports extended 8-bit ASCII (ISO-8859-1); it does not support UTF-8 or other locale-specific character sets.

      Specified by:
      readLine in interface DataInput
      Returns:
      the next line of text from this input stream.
      Throws:
      IOException - if an I/O error occurs.

    • readUTF8Line

      public final String readUTF8Line() throws IOException
      This method reads a line of text encoded in UTF-8, terminated by a line ending (CR, LF, or CRLF) or EOF.

      Note that this method only supports UTF-8; it does not support other locale-specific character sets.

      Returns:
      the next line of text from this input stream.
      Throws:
      IOException - if an I/O error occurs.

    • readUTF

      public final String readUTF() throws IOException
      See the general contract of the readUTF method of DataInput.
      Specified by:
      readUTF in interface DataInput
      Returns:
      a Unicode string.
      Throws:
      EOFException - if this input stream reaches the end before reading all the bytes.
      IOException - the stream has been closed and the contained input stream does not support reading after close, or another I/O error occurs.
      UTFDataFormatException - if the bytes do not represent a valid modified UTF-8 encoding of a string.

    • getSeqNum

      public int getSeqNum()
      Get the identifying sequence number of this instance of NonshareableDoubleBufferedDataInputStream.
      Returns:
      count of how many NonshareableDoubleBufferedDataInputStream were created

    • isEof

      public boolean isEof()
      Test if this stream's file-reading thread has reached end-of-file of the underlying stream. Note that this does not imply that there are no unread buffered bytes left in this stream; getting a -1 from read() will indicate that.
      Returns:
      booloean true if the underlying stream's end-of-file has been reached

    • getNumBufNotReady

      public int getNumBufNotReady()
      Report the number of times the file-reading thread had not provided sufficient data for the data-consuming thread when it wanted more data.
      Returns:
      number of times the readXXX methods had to wait (block) for more data
      See Also:

    • getTimeWaitingForBuf

      public long getTimeWaitingForBuf()
      Return the total amount of time (in milliseconds) that the data-consuming thread had to wait because the file-reading thread wasn't fast enough.
      Returns:
      total stall time in milliseconds
      See Also:

    • getByteCount

      public long getByteCount()
      Get the total number of bytes read through this stream (so far).
      Returns:
      count of bytes read from this stream