Module Limiter

Implements a token-bucket-based throttling rate limiter. This module is useful for limiting network clients to a sensible query rate, or in any case where you have jobs that consume a scarce but replenishable resource.

In a standard token bucket there is an infinite incoming supply of tokens that fill a single bucket.

This version implements a closed system where tokens move through three possible states:

Tokens "drop" from the hopper into the bucket at a set rate, and can be taken from the bucket by clients and put into flight. Once the client is finished with whatever tokens are required for its task, it is responsible for moving them from "in flight" back into the hopper.

Most use cases are covered by the Token_bucket, Throttle, and Throttled_rate_limiter modules, but the Expert module provides full access to the module internals.

This interface is the simple, non-concurrent interface, and requires machinery on top to implement a specific strategy. See Limiter_async for an async-friendly implementation on top of this module.

Most functions in this interface take an explicit time as an argument. now is expected to be monotonically increasing. now's that are set in the past are effectively moved up to the current time of the bucket.

type t
val sexp_of_t : t -> Ppx_sexp_conv_lib.Sexp.t
type limiter = t
val sexp_of_limiter : limiter -> Ppx_sexp_conv_lib.Sexp.t
module Infinite_or_finite : sig ... end
module Try_take_result : sig ... end
module Try_return_to_bucket_result : sig ... end
module Tokens_may_be_available_result : sig ... end
module Try_increase_bucket_limit_result : sig ... end
module Token_bucket : sig ... end

Implements a basic token-bucket-based rate limiter. Users of the throttle must successfully call try_take before doing work.

module Throttle : sig ... end

Implements a basic throttle. Users of the throttle must successfully call start_job before beginning work and must call finish_job once, and only once, when a job is completed.

module Throttled_rate_limiter : sig ... end

A Throttled_rate_limiter combines a Token_bucket and a Throttle. Unlike a Token_bucket, jobs cannot consume variable numbers of tokens, but the number of outstanding jobs is also limited to max_concurrent_jobs. Like a Throttle, finish_job must be called once, and only once, when a job is completed.

Common read-only operations

val bucket_limit : t -> int
val in_bucket : t -> now:Core_kernel.Time_ns.t -> int

Tokens available to immediately take.

val in_hopper : t -> now:Core_kernel.Time_ns.t -> int Infinite_or_finite.t

Tokens waiting to drop at the hopper_to_bucket_rate_per_sec.

val in_flight : t -> now:Core_kernel.Time_ns.t -> int

Tokens that have been taken, but not yet returned.

val in_limiter : t -> now:Core_kernel.Time_ns.t -> int Infinite_or_finite.t

Total number of tokens in the limiter in_hopper + in_bucket.

val in_system : t -> now:Core_kernel.Time_ns.t -> int Infinite_or_finite.t

Total number of tokens in the entire system in_hopper + in_bucket + in_flight.

val hopper_to_bucket_rate_per_sec : t -> float Infinite_or_finite.t

Note that this isn't guaranteed to be equal to the rate_per_sec that was passed in to the constructor, due to floating point error.

module Expert : sig ... end

Expert operations.

include Core_kernel.Invariant.S with type t := t
val invariant : t -> unit