Package com.yahoo.docproc.impl

Class DocprocService

java.lang.Object
com.yahoo.component.AbstractComponent
com.yahoo.docproc.impl.DocprocService
All Implemented Interfaces:
com.yahoo.component.Component, com.yahoo.component.Deconstructable, Comparable<com.yahoo.component.Component>
public class DocprocService extends com.yahoo.component.AbstractComponent

The document processing service. Use this to set up a document processing chain and to process documents using that chain. Note that there may be multiple named instances of this service in the same runtime. The default service is called "default" and is always present.

To create a server which receives documents from the network and processes them, have a look at com.yahoo.docproc.server.Server.

This class is thread safe.

Author:
bratseth

  • Field Summary

    Fields
    Modifier and Type
    Field
    Description
    static SchemaMap
    schemaMap
     

    Fields inherited from class com.yahoo.component.AbstractComponent

    isDeconstructable

  • Constructor Summary

    Constructors
    Constructor
    Description
    DocprocService(com.yahoo.component.ComponentId id)
     
    DocprocService(com.yahoo.component.ComponentId id, CallStack stack, com.yahoo.document.DocumentTypeManager mgr, int numThreads)
    Creates a new docproc service, which is set to be in service.
    DocprocService(String name)
    Creates a service with a name with an unbounded input queue.

  • Method Summary

    Modifier and Type
    Method
    Description
    void
    deconstruct()
     
    boolean
    doWork()
    Do some work in this service.
    boolean
    doWorkBlocking()
    Do some work in this service.
    CallStack
    getCallStack()
    Returns the processing chain of this service.
    com.yahoo.document.DocumentTypeManager
    getDocumentTypeManager()
     
    DocprocExecutor
    getExecutor()
    Returns the DocprocExecutor of this DocprocService.
    String
    getName()
     
    int
    getQueueSize()
     
    ThreadPoolExecutor
    getThreadPoolExecutor()
     
    boolean
    isAcceptingNewProcessings()
    Returns true if this service currently accepts new incoming processings via process(...). Default is true.
    boolean
    isInService()
    Returns true if this is currently processing incoming processings (in service), or false if they are just queued up (out of service).
    void
    process(Processing processing)
    Asynchronously process the given Processing using the processing chain of this service
    void
    process(Processing processing, ProcessingEndpoint endp)
    Asynchronously process the given Processing using the processing chain of this service, and call the specified ProcessingEndpoint when done.
    void
    process(com.yahoo.document.DocumentOperation documentOperation)
    Asynchronously process the given document operation using the processing chain of this service.
    void
    process(com.yahoo.document.DocumentOperation documentOperation, ProcessingEndpoint endp)
    Asynchronously process the given document put or document update using the processing chain of this service, and call the specified ProcessingEndpoint when done.
    void
    processDocumentOperations(List<com.yahoo.document.DocumentOperation> documentOperations)
    Asynchronously process the given document operations as one unit using the processing chain of this service.
    void
    processDocumentOperations(List<com.yahoo.document.DocumentOperation> documentOperations, ProcessingEndpoint endp)
    Asynchronously process the given document operations as one unit using the processing chain of this service, and call the specified ProcessingEndpoint when done.
    void
    setAcceptingNewProcessings(boolean acceptingNewProcessings)
    Sets whether this service should accept new incoming processings via process(...).
    void
    setCallStack(CallStack stack)
    Sets a new processing stack for this service.
    void
    setDocumentTypeManager(com.yahoo.document.DocumentTypeManager documentTypeManager)
     
    void
    setInService(boolean inService)
    Sets whether this should currently perform any processing.

    Methods inherited from class com.yahoo.component.AbstractComponent

    clone, compareTo, getClassName, getId, getIdString, hasInitializedId, initId, isDeconstructable, setIsDeconstructable, toString

    Methods inherited from class java.lang.Object

    equals, finalize, getClass, hashCode, notify, notifyAll, wait, wait, wait

  • Field Details

    • schemaMap

      public static SchemaMap schemaMap

  • Constructor Details

    • DocprocService

      public DocprocService(com.yahoo.component.ComponentId id)

    • DocprocService

      public DocprocService(com.yahoo.component.ComponentId id, CallStack stack, com.yahoo.document.DocumentTypeManager mgr, int numThreads)
      Creates a new docproc service, which is set to be in service.
      Parameters:
      id - the component id of the new service.
      stack - the call stack to use.
      mgr - the document type manager to use.
      numThreads - to have in the thread pool

    • DocprocService

      public DocprocService(String name)
      Creates a service with a name with an unbounded input queue. If the given name is null or the empty string, it will become the name "default". Testing only

  • Method Details

    • deconstruct

      public void deconstruct()
      Specified by:
      deconstruct in interface com.yahoo.component.Deconstructable
      Overrides:
      deconstruct in class com.yahoo.component.AbstractComponent

    • getDocumentTypeManager

      public com.yahoo.document.DocumentTypeManager getDocumentTypeManager()

    • setDocumentTypeManager

      public void setDocumentTypeManager(com.yahoo.document.DocumentTypeManager documentTypeManager)

    • getQueueSize

      public int getQueueSize()

    • getExecutor

      public DocprocExecutor getExecutor()
      Returns the DocprocExecutor of this DocprocService. This can be used to synchronously process one Processing.
      Returns:
      the DocprocExecutor of this DocprocService, or null if a CallStack has not yet been set.

    • getThreadPoolExecutor

      public ThreadPoolExecutor getThreadPoolExecutor()

    • setInService

      public void setInService(boolean inService)
      Sets whether this should currently perform any processing. New processings will be accepted also when this is out of service, but no processing will happen when it is out of service.

    • isInService

      public boolean isInService()
      Returns true if this is currently processing incoming processings (in service), or false if they are just queued up (out of service). By default, this is out of service.

    • isAcceptingNewProcessings

      public boolean isAcceptingNewProcessings()
      Returns true if this service currently accepts new incoming processings via process(...). Default is true.
      Returns:
      true if accepting new incoming processings

    • setAcceptingNewProcessings

      public void setAcceptingNewProcessings(boolean acceptingNewProcessings)
      Sets whether this service should accept new incoming processings via process(...).
      Parameters:
      acceptingNewProcessings - true if service should accept new incoming processings

    • getName

      public String getName()

    • getCallStack

      public CallStack getCallStack()
      Returns the processing chain of this service. This stack can not be modified. To change the stack, set a new one.

    • setCallStack

      public void setCallStack(CallStack stack)
      Sets a new processing stack for this service. This will be the Prototype for the call stacks of individual processings in this service

    • process

      public void process(Processing processing, ProcessingEndpoint endp)
      Asynchronously process the given Processing using the processing chain of this service, and call the specified ProcessingEndpoint when done.
      Throws:
      RuntimeException - caused by a QueueFullException if this DocprocService has a bounded input queue and the queue is full
      IllegalStateException - if this DocprocService is not accepting new incoming processings

    • process

      public void process(Processing processing)
      Asynchronously process the given Processing using the processing chain of this service
      Throws:
      RuntimeException - caused by a QueueFullException if this DocprocService has a bounded input queue and the queue is full
      IllegalStateException - if this DocprocService is not accepting new incoming processings

    • process

      public void process(com.yahoo.document.DocumentOperation documentOperation, ProcessingEndpoint endp)
      Asynchronously process the given document put or document update using the processing chain of this service, and call the specified ProcessingEndpoint when done.
      Throws:
      RuntimeException - caused by a QueueFullException if this DocprocService has a bounded input queue and the queue is full
      IllegalStateException - if this DocprocService is not accepting new incoming processings

    • process

      public void process(com.yahoo.document.DocumentOperation documentOperation)
      Asynchronously process the given document operation using the processing chain of this service.
      Throws:
      RuntimeException - caused by a QueueFullException if this DocprocService has a bounded input queue and the queue is full
      IllegalStateException - if this DocprocService is not accepting new incoming processings

    • processDocumentOperations

      public void processDocumentOperations(List<com.yahoo.document.DocumentOperation> documentOperations, ProcessingEndpoint endp)
      Asynchronously process the given document operations as one unit using the processing chain of this service, and call the specified ProcessingEndpoint when done.
      Throws:
      RuntimeException - caused by a QueueFullException if this DocprocService has a bounded input queue and the queue is full
      IllegalStateException - if this DocprocService is not accepting new incoming processings

    • processDocumentOperations

      public void processDocumentOperations(List<com.yahoo.document.DocumentOperation> documentOperations)
      Asynchronously process the given document operations as one unit using the processing chain of this service.
      Throws:
      RuntimeException - caused by a QueueFullException if this DocprocService has a bounded input queue and the queue is full
      IllegalStateException - if this DocprocService is not accepting new incoming processings

    • doWork

      public boolean doWork()

      Do some work in this service. This will perform some processing and return in a "short" while, as long as individual processors also returns.

      This method is thread safe - multiple threads may call doWork at any time. Note that processors should be non-blocking, so multiple threads should be used primarily to utilize multiple processors.

      Returns:
      true if some work was performed, false if no work could be performed at this time, because there are no outstanding processings, or because this is out of service. Note that since processings may arrive or be put back by another thread at any time, this return value does not mean that no work will be available if doWork as called again immediately.

    • doWorkBlocking

      public boolean doWorkBlocking() throws InterruptedException

      Do some work in this service. This will perform some processing and return in a "short" while, as long as individual processors also returns. Note that if the internal queue is empty when this method is called, it will block until some work is submitted by a call to process() by another thread.

      This method is thread safe - multiple threads may call doWorkBlocking at any time. Note that processors should be non-blocking, so multiple threads should be used primarily to utilize multiple processors.

      Returns:
      always true, since if the internal queue is empty when this method is called, it will block until some work is submitted by a call to process() by another thread.
      Throws:
      InterruptedException - if a call to this method is interrupted while waiting for data to become available