Class/Object

org.apache.spark.memory

SnappyUnifiedMemoryManager

Related Docs: object SnappyUnifiedMemoryManager | package memory

Permalink

class SnappyUnifiedMemoryManager extends UnifiedMemoryManager with StoreUnifiedManager

When there is request for execution or storage memory, critical up and eviction up events are checked. If they are set, try to free the memory cached by Spark rdds by calling memoryStore.evictBlocksToFreeSpace. If enough memory cannot be freed, return the call and let Spark take a corrective action. In such cases Spark either fails the task or move the current RDDs data to disk. If the critical and eviction events are not set, it asks the UnifiedMemoryManager to allocate the space.

Linear Supertypes
StoreUnifiedManager, UnifiedMemoryManager, MemoryManager, internal.Logging, AnyRef, Any
Ordering
  1. Alphabetic
  2. By Inheritance
Inherited
  1. SnappyUnifiedMemoryManager
  2. StoreUnifiedManager
  3. UnifiedMemoryManager
  4. MemoryManager
  5. Logging
  6. AnyRef
  7. Any
  1. Hide All
  2. Show All
Visibility
  1. Public
  2. All

Instance Constructors

  1. new SnappyUnifiedMemoryManager(conf: SparkConf, numCores: Int)

    Permalink

  2. new SnappyUnifiedMemoryManager(conf: SparkConf, numCores: Int, tempManager: Boolean = false)

    Permalink

Value Members

  1. final def !=(arg0: Any): Boolean

    Permalink
    Definition Classes
    AnyRef → Any

  2. final def ##(): Int

    Permalink
    Definition Classes
    AnyRef → Any

  3. final def ==(arg0: Any): Boolean

    Permalink
    Definition Classes
    AnyRef → Any

  4. val SPARK_CACHE: String

    Permalink

  5. def acquireStorageMemory(blockId: BlockId, numBytes: Long, memoryMode: MemoryMode): Boolean

    Permalink
    Definition Classes
    SnappyUnifiedMemoryManager → UnifiedMemoryManager → MemoryManager

  6. def acquireStorageMemoryForObject(objectName: String, blockId: BlockId, numBytes: Long, memoryMode: MemoryMode, buffer: UMMMemoryTracker, shouldEvict: Boolean): Boolean

    Permalink
    Definition Classes
    SnappyUnifiedMemoryManager → StoreUnifiedManager

  7. def acquireUnrollMemory(blockId: BlockId, numBytes: Long, memoryMode: MemoryMode): Boolean

    Permalink
    Definition Classes
    UnifiedMemoryManager → MemoryManager

  8. final def asInstanceOf[T0]: T0

    Permalink
    Definition Classes
    Any

  9. def assertInvariants(): Unit

    Permalink
    Attributes
    protected
    Definition Classes
    UnifiedMemoryManager

  10. def changeOffHeapOwnerToStorage(buffer: ByteBuffer, allowNonAllocator: Boolean): Unit

    Permalink
    Definition Classes
    SnappyUnifiedMemoryManager → StoreUnifiedManager

  11. def clone(): AnyRef

    Permalink
    Attributes
    protected[java.lang]
    Definition Classes
    AnyRef
    Annotations
    @throws( ... )

  12. def dropStorageMemoryForObject(name: String, memoryMode: MemoryMode, ignoreNumBytes: Long): Long

    Permalink
    Definition Classes
    SnappyUnifiedMemoryManager → StoreUnifiedManager

  13. final def eq(arg0: AnyRef): Boolean

    Permalink
    Definition Classes
    AnyRef

  14. def equals(arg0: Any): Boolean

    Permalink
    Definition Classes
    AnyRef → Any

  15. final def executionMemoryUsed: Long

    Permalink
    Definition Classes
    MemoryManager

  16. def finalize(): Unit

    Permalink
    Attributes
    protected[java.lang]
    Definition Classes
    AnyRef
    Annotations
    @throws( classOf[java.lang.Throwable] )

  17. final def getClass(): Class[_]

    Permalink
    Definition Classes
    AnyRef → Any

  18. def getExecutionPoolSize(memoryMode: MemoryMode): Long

    Permalink
    Definition Classes
    SnappyUnifiedMemoryManager → StoreUnifiedManager

  19. def getExecutionPoolUsedMemory(memoryMode: MemoryMode): Long

    Permalink
    Definition Classes
    SnappyUnifiedMemoryManager → StoreUnifiedManager

  20. def getOffHeapMemory(objectName: String): Long

    Permalink
    Definition Classes
    SnappyUnifiedMemoryManager → StoreUnifiedManager

  21. def getStoragePoolMemoryUsed(memoryMode: MemoryMode): Long

    Permalink
    Definition Classes
    SnappyUnifiedMemoryManager → StoreUnifiedManager

  22. def getStoragePoolSize(memoryMode: MemoryMode): Long

    Permalink
    Definition Classes
    SnappyUnifiedMemoryManager → StoreUnifiedManager

  23. def hasOffHeap: Boolean

    Permalink
    Definition Classes
    SnappyUnifiedMemoryManager → StoreUnifiedManager

  24. def hashCode(): Int

    Permalink
    Definition Classes
    AnyRef → Any

  25. def initializeLogIfNecessary(isInterpreter: Boolean): Unit

    Permalink
    Attributes
    protected
    Definition Classes
    Logging

  26. final def isDebugEnabled: Boolean

    Permalink
    Attributes
    protected
    Definition Classes
    Logging

  27. final def isInfoEnabled: Boolean

    Permalink
    Attributes
    protected
    Definition Classes
    Logging

  28. final def isInstanceOf[T0]: Boolean

    Permalink
    Definition Classes
    Any

  29. final def isTraceEnabled: Boolean

    Permalink
    Attributes
    protected
    Definition Classes
    Logging

  30. def log: Logger

    Permalink
    Attributes
    protected
    Definition Classes
    Logging

  31. def logDebug(msg: ⇒ String, throwable: Throwable): Unit

    Permalink
    Attributes
    protected
    Definition Classes
    Logging

  32. def logDebug(msg: ⇒ String): Unit

    Permalink
    Attributes
    protected
    Definition Classes
    Logging

  33. def logError(msg: ⇒ String, throwable: Throwable): Unit

    Permalink
    Attributes
    protected
    Definition Classes
    Logging

  34. def logError(msg: ⇒ String): Unit

    Permalink
    Attributes
    protected
    Definition Classes
    Logging

  35. def logInfo(msg: ⇒ String, throwable: Throwable): Unit

    Permalink
    Attributes
    protected
    Definition Classes
    Logging

  36. def logInfo(msg: ⇒ String): Unit

    Permalink
    Attributes
    protected
    Definition Classes
    Logging

  37. def logName: String

    Permalink
    Attributes
    protected
    Definition Classes
    Logging

  38. def logStats(): Unit

    Permalink
    Definition Classes
    SnappyUnifiedMemoryManager → StoreUnifiedManager

  39. def logTrace(msg: ⇒ String, throwable: Throwable): Unit

    Permalink
    Attributes
    protected
    Definition Classes
    Logging

  40. def logTrace(msg: ⇒ String): Unit

    Permalink
    Attributes
    protected
    Definition Classes
    Logging

  41. def logWarning(msg: ⇒ String, throwable: Throwable): Unit

    Permalink
    Attributes
    protected
    Definition Classes
    Logging

  42. def logWarning(msg: ⇒ String): Unit

    Permalink
    Attributes
    protected
    Definition Classes
    Logging

  43. val maxHeapMemory: Long

    Permalink

    the maximum heap memory that is available for use by MemoryManager; callers should leave out some amount of "reserved memory" for unaccounted object allocations

    the maximum heap memory that is available for use by MemoryManager; callers should leave out some amount of "reserved memory" for unaccounted object allocations

    Definition Classes
    SnappyUnifiedMemoryManager → UnifiedMemoryManager

  44. val maxOffHeapMemory: Long

    Permalink
    Attributes
    protected[this]
    Definition Classes
    MemoryManager

  45. def maxOffHeapStorageMemory: Long

    Permalink
    Definition Classes
    UnifiedMemoryManager → MemoryManager

  46. def maxOnHeapStorageMemory: Long

    Permalink
    Definition Classes
    UnifiedMemoryManager → MemoryManager

  47. final def ne(arg0: AnyRef): Boolean

    Permalink
    Definition Classes
    AnyRef

  48. final def notify(): Unit

    Permalink
    Definition Classes
    AnyRef

  49. final def notifyAll(): Unit

    Permalink
    Definition Classes
    AnyRef

  50. val offHeapExecutionMemoryPool: ExecutionMemoryPool

    Permalink
    Attributes
    protected
    Definition Classes
    MemoryManager

  51. val offHeapStorageMemory: Long

    Permalink
    Attributes
    protected[this]
    Definition Classes
    MemoryManager

  52. val offHeapStorageMemoryPool: StorageMemoryPool

    Permalink
    Attributes
    protected
    Definition Classes
    MemoryManager

  53. val onHeapExecutionMemoryPool: ExecutionMemoryPool

    Permalink
    Attributes
    protected
    Definition Classes
    MemoryManager

  54. val onHeapStorageMemoryPool: StorageMemoryPool

    Permalink
    Attributes
    protected
    Definition Classes
    MemoryManager

  55. val pageSizeBytes: Long

    Permalink
    Definition Classes
    MemoryManager

  56. final def releaseAllStorageMemory(): Unit

    Permalink
    Definition Classes
    MemoryManager

  57. def releaseStorageMemory(numBytes: Long, memoryMode: MemoryMode): Unit

    Permalink
    Definition Classes
    SnappyUnifiedMemoryManager → MemoryManager

  58. def releaseStorageMemoryForObject(objectName: String, numBytes: Long, memoryMode: MemoryMode): Unit

    Permalink
    Definition Classes
    SnappyUnifiedMemoryManager → StoreUnifiedManager

  59. final def releaseUnrollMemory(numBytes: Long, memoryMode: MemoryMode): Unit

    Permalink
    Definition Classes
    MemoryManager

  60. final def setMemoryStore(store: MemoryStore): Unit

    Permalink
    Definition Classes
    MemoryManager

  61. def shouldStopRecovery(): Boolean

    Permalink
    Definition Classes
    SnappyUnifiedMemoryManager → StoreUnifiedManager

  62. final def storageMemoryUsed: Long

    Permalink
    Definition Classes
    MemoryManager

  63. final def synchronized[T0](arg0: ⇒ T0): T0

    Permalink
    Definition Classes
    AnyRef

  64. val tempManager: Boolean

    Permalink

  65. val threadsWaitingForStorage: AtomicInteger

    Permalink

  66. def toString(): String

    Permalink
    Definition Classes
    AnyRef → Any

  67. def tryExplicitGC(): Unit

    Permalink

  68. final val tungstenMemoryMode: MemoryMode

    Permalink
    Definition Classes
    MemoryManager

  69. final def wait(): Unit

    Permalink
    Definition Classes
    AnyRef
    Annotations
    @throws( ... )

  70. final def wait(arg0: Long, arg1: Int): Unit

    Permalink
    Definition Classes
    AnyRef
    Annotations
    @throws( ... )

  71. final def wait(arg0: Long): Unit

    Permalink
    Definition Classes
    AnyRef
    Annotations
    @throws( ... )

Inherited from StoreUnifiedManager

Inherited from UnifiedMemoryManager

Inherited from MemoryManager

Inherited from internal.Logging

Inherited from AnyRef

Inherited from Any

Ungrouped