Package com.yahoo.vespa.hadoop.pig

Class VespaStorage

  • All Implemented Interfaces:
    org.apache.pig.StoreFuncInterface
    public class VespaStorage
extends org.apache.pig.StoreFunc
    A small Pig UDF wrapper around the Vespa http client for feeding data into a Vespa endpoint.
    Author:
    lesters

    • Constructor Summary

      Constructors 
      Constructor Description
      VespaStorage()  
      VespaStorage​(java.lang.String... params)  

    • Method Summary

      All Methods Static Methods Instance Methods Concrete Methods 
      Modifier and Type Method Description
      static <T extends java.io.Serializable>
      T      		 base64Deserialize​(java.lang.String s)  
      static java.lang.String base64Serialize​(java.lang.Object o)  
      void checkSchema​(org.apache.pig.ResourceSchema resourceSchema)  
      void cleanupOnFailure​(java.lang.String s, org.apache.hadoop.mapreduce.Job job)  
      void cleanupOnSuccess​(java.lang.String s, org.apache.hadoop.mapreduce.Job job)  
      org.apache.hadoop.mapreduce.OutputFormat getOutputFormat()  
      void prepareToWrite​(org.apache.hadoop.mapreduce.RecordWriter recordWriter)  
      void putNext​(org.apache.pig.data.Tuple tuple)  
      java.lang.String relToAbsPathForStoreLocation​(java.lang.String endpoint, org.apache.hadoop.fs.Path path)  
      void setStoreFuncUDFContextSignature​(java.lang.String s)  
      void setStoreLocation​(java.lang.String endpoint, org.apache.hadoop.mapreduce.Job job)  

      • Methods inherited from class org.apache.pig.StoreFunc

        cleanupOnFailureImpl, warn

      • Methods inherited from class java.lang.Object

        clone, equals, finalize, getClass, hashCode, notify, notifyAll, toString, wait, wait, wait

    • Constructor Detail

      • VespaStorage

        public VespaStorage()

      • VespaStorage

        public VespaStorage​(java.lang.String... params)

    • Method Detail

      • getOutputFormat

        public org.apache.hadoop.mapreduce.OutputFormat getOutputFormat()
                                                         throws java.io.IOException
        Specified by:
        getOutputFormat in interface org.apache.pig.StoreFuncInterface
        Specified by:
        getOutputFormat in class org.apache.pig.StoreFunc
        Throws:
        java.io.IOException

      • setStoreLocation

        public void setStoreLocation​(java.lang.String endpoint,
                             org.apache.hadoop.mapreduce.Job job)
                      throws java.io.IOException
        Specified by:
        setStoreLocation in interface org.apache.pig.StoreFuncInterface
        Specified by:
        setStoreLocation in class org.apache.pig.StoreFunc
        Throws:
        java.io.IOException

      • prepareToWrite

        public void prepareToWrite​(org.apache.hadoop.mapreduce.RecordWriter recordWriter)
                    throws java.io.IOException
        Specified by:
        prepareToWrite in interface org.apache.pig.StoreFuncInterface
        Specified by:
        prepareToWrite in class org.apache.pig.StoreFunc
        Throws:
        java.io.IOException

      • putNext

        public void putNext​(org.apache.pig.data.Tuple tuple)
             throws java.io.IOException
        Specified by:
        putNext in interface org.apache.pig.StoreFuncInterface
        Specified by:
        putNext in class org.apache.pig.StoreFunc
        Throws:
        java.io.IOException

      • checkSchema

        public void checkSchema​(org.apache.pig.ResourceSchema resourceSchema)
                 throws java.io.IOException
        Specified by:
        checkSchema in interface org.apache.pig.StoreFuncInterface
        Overrides:
        checkSchema in class org.apache.pig.StoreFunc
        Throws:
        java.io.IOException

      • relToAbsPathForStoreLocation

        public java.lang.String relToAbsPathForStoreLocation​(java.lang.String endpoint,
                                                     org.apache.hadoop.fs.Path path)
                                              throws java.io.IOException
        Specified by:
        relToAbsPathForStoreLocation in interface org.apache.pig.StoreFuncInterface
        Overrides:
        relToAbsPathForStoreLocation in class org.apache.pig.StoreFunc
        Throws:
        java.io.IOException

      • setStoreFuncUDFContextSignature

        public void setStoreFuncUDFContextSignature​(java.lang.String s)
        Specified by:
        setStoreFuncUDFContextSignature in interface org.apache.pig.StoreFuncInterface
        Overrides:
        setStoreFuncUDFContextSignature in class org.apache.pig.StoreFunc

      • cleanupOnFailure

        public void cleanupOnFailure​(java.lang.String s,
                             org.apache.hadoop.mapreduce.Job job)
                      throws java.io.IOException
        Specified by:
        cleanupOnFailure in interface org.apache.pig.StoreFuncInterface
        Overrides:
        cleanupOnFailure in class org.apache.pig.StoreFunc
        Throws:
        java.io.IOException

      • cleanupOnSuccess

        public void cleanupOnSuccess​(java.lang.String s,
                             org.apache.hadoop.mapreduce.Job job)
                      throws java.io.IOException
        Specified by:
        cleanupOnSuccess in interface org.apache.pig.StoreFuncInterface
        Overrides:
        cleanupOnSuccess in class org.apache.pig.StoreFunc
        Throws:
        java.io.IOException

      • base64Serialize

        public static java.lang.String base64Serialize​(java.lang.Object o)
                                        throws java.io.IOException
        Throws:
        java.io.IOException

      • base64Deserialize

        public static <T extends java.io.Serializable> T base64Deserialize​(java.lang.String s)
                                                            throws java.io.IOException
        Throws:
        java.io.IOException