org.apache.spark.streaming.receiver

Class RateLimiter

Object

org.apache.spark.streaming.receiver.RateLimiter

All Implemented Interfaces:

Logging

Direct Known Subclasses:

BlockGenerator

public abstract class RateLimiter
extends Object
implements Logging

Provides waitToPush() method to limit the rate at which receivers consume data.

waitToPush method will block the thread if too many messages have been pushed too quickly, and only return when a new message has been pushed. It assumes that only one message is pushed at a time.

The spark configuration spark.streaming.receiver.maxRate gives the maximum number of messages per second that each receiver will accept.

Constructor Summary

Constructors

Constructor and Description
RateLimiter(SparkConf conf)

Method Summary

Methods

Modifier and Type	Method and Description
void	waitToPush()

Methods inherited from class Object

equals, getClass, hashCode, notify, notifyAll, toString, wait, wait, wait

Methods inherited from interface org.apache.spark.Logging

initializeIfNecessary, initializeLogging, isTraceEnabled, log_, log, logDebug, logDebug, logError, logError, logInfo, logInfo, logName, logTrace, logTrace, logWarning, logWarning

Constructor Detail

RateLimiter

public RateLimiter(SparkConf conf)

Method Detail

waitToPush

public void waitToPush()