Class CqlRecordReader

  • All Implemented Interfaces:
    java.io.Closeable, java.lang.AutoCloseable, org.apache.hadoop.mapred.RecordReader<java.lang.Long,​com.datastax.driver.core.Row>

    public class CqlRecordReader
    extends org.apache.hadoop.mapreduce.RecordReader<java.lang.Long,​com.datastax.driver.core.Row>
    implements org.apache.hadoop.mapred.RecordReader<java.lang.Long,​com.datastax.driver.core.Row>, java.lang.AutoCloseable

    CqlRecordReader reads the rows return from the CQL query It uses CQL auto-paging.

    Return a Long as a local CQL row key starts from 0;

    Row as C* java driver CQL result set row 1) select clause must include partition key columns (to calculate the progress based on the actual CF row processed) 2) where clause must include token(partition_key1, ... , partition_keyn) > ? and token(partition_key1, ... , partition_keyn) <= ? (in the right order)
    • Field Detail

      • nativeProtocolVersion

        protected int nativeProtocolVersion
    • Constructor Detail

      • CqlRecordReader

        public CqlRecordReader()
    • Method Detail

      • initialize

        public void initialize​(org.apache.hadoop.mapreduce.InputSplit split,
                               org.apache.hadoop.mapreduce.TaskAttemptContext context)
                        throws java.io.IOException
        Specified by:
        initialize in class org.apache.hadoop.mapreduce.RecordReader<java.lang.Long,​com.datastax.driver.core.Row>
        Throws:
        java.io.IOException
      • close

        public void close()
        Specified by:
        close in interface java.lang.AutoCloseable
        Specified by:
        close in interface java.io.Closeable
        Specified by:
        close in interface org.apache.hadoop.mapred.RecordReader<java.lang.Long,​com.datastax.driver.core.Row>
        Specified by:
        close in class org.apache.hadoop.mapreduce.RecordReader<java.lang.Long,​com.datastax.driver.core.Row>
      • getCurrentKey

        public java.lang.Long getCurrentKey()
        Specified by:
        getCurrentKey in class org.apache.hadoop.mapreduce.RecordReader<java.lang.Long,​com.datastax.driver.core.Row>
      • getCurrentValue

        public com.datastax.driver.core.Row getCurrentValue()
        Specified by:
        getCurrentValue in class org.apache.hadoop.mapreduce.RecordReader<java.lang.Long,​com.datastax.driver.core.Row>
      • getProgress

        public float getProgress()
        Specified by:
        getProgress in interface org.apache.hadoop.mapred.RecordReader<java.lang.Long,​com.datastax.driver.core.Row>
        Specified by:
        getProgress in class org.apache.hadoop.mapreduce.RecordReader<java.lang.Long,​com.datastax.driver.core.Row>
      • nextKeyValue

        public boolean nextKeyValue()
                             throws java.io.IOException
        Specified by:
        nextKeyValue in class org.apache.hadoop.mapreduce.RecordReader<java.lang.Long,​com.datastax.driver.core.Row>
        Throws:
        java.io.IOException
      • next

        public boolean next​(java.lang.Long key,
                            com.datastax.driver.core.Row value)
                     throws java.io.IOException
        Specified by:
        next in interface org.apache.hadoop.mapred.RecordReader<java.lang.Long,​com.datastax.driver.core.Row>
        Throws:
        java.io.IOException
      • getPos

        public long getPos()
                    throws java.io.IOException
        Specified by:
        getPos in interface org.apache.hadoop.mapred.RecordReader<java.lang.Long,​com.datastax.driver.core.Row>
        Throws:
        java.io.IOException
      • createKey

        public java.lang.Long createKey()
        Specified by:
        createKey in interface org.apache.hadoop.mapred.RecordReader<java.lang.Long,​com.datastax.driver.core.Row>
      • createValue

        public com.datastax.driver.core.Row createValue()
        Specified by:
        createValue in interface org.apache.hadoop.mapred.RecordReader<java.lang.Long,​com.datastax.driver.core.Row>
      • getNativeProtocolVersion

        public int getNativeProtocolVersion()
        Return native version protocol of the cluster connection
        Returns:
        serialization protocol version.