Kafka: TimeoutException: Expiring N record(s) for topic-partition

Common symptoms

• Records your application clearly sent never appear on the topic, with no exception visible anywhere in your own logs
• A downstream consumer reports gaps in a sequence that should be contiguous
• Producer throughput degrades gradually before failures start, rather than failing immediately
• The failure clusters around deploys, network blips, or broker maintenance windows

The exact error string

org.apache.kafka.common.errors.TimeoutException: Expiring 15 record(s) for
orders-3:120000 ms has passed since batch creation

Read the number in the message carefully — it's easy to misinterpret. 120000 ms is not how long the specific failure took; it is the value of delivery.timeout.ms, the configured ceiling. The batch was created, sat in the buffer, and once that much time had passed with no successful acknowledgement, the producer gave up on every record in it.

A related but distinct producer failure is worth telling apart from this one, since the fix is different:

org.apache.kafka.common.errors.TimeoutException: Failed to allocate memory within
the configured max blocking time 60000 ms.
org.apache.kafka.clients.producer.BufferExhaustedException

This one fires from send() itself, before a batch is ever created — the producer's buffer.memory is already full and max.block.ms elapsed while waiting for space to free up. It means the buffer is saturated right now, whereas "Expiring N record(s)" means a batch that did get buffered still couldn't get out the door in time. Both point at the same underlying problem — the producer generating faster than the network or brokers can absorb — but this variant surfaces earlier and more abruptly. The buffer-saturation guidance below (Fix 3) applies to both.

What delivery.timeout.ms actually covers

This single setting bounds the entire journey a record takes from your call to send() to a broker's acknowledgement: time waiting in the buffer for a batch to fill, time spent in-flight to the broker, and time spent on retries after a retriable failure. It does not reset on each retry — it is a hard ceiling on the record's total time in the producer, from creation to either success or expiry.

delivery.timeout.ms=120000   # default: 2 minutes, total budget per record
request.timeout.ms=30000     # default: per-request wait for a broker reply
retries=2147483647            # default (with idempotence on): effectively unlimited

# delivery.timeout.ms must be >= request.timeout.ms + linger.ms, and the
# producer refuses to start otherwise

That constraint matters when tuning: raising request.timeout.ms without also raising delivery.timeout.ms either fails validation or leaves less room for retries than you think.

Fix 1: confirm the broker is reachable

This is the most common cause and the fastest to rule out. Check whether the producer can connect to the addresses in bootstrap.servers at all:

kafka-broker-api-versions.sh --bootstrap-server broker1:9092,broker2:9092

# a reachable broker prints its supported API versions:
# broker1:9092 (id: 1 rack: null) -> (
#   Produce(0): 0 to 9 [usable: 9],
#   ...
# )
#
# an unreachable one hangs, then prints a connection error instead

If this fails, the fix is networking, not Kafka configuration: a firewall rule, a wrong hostname (especially in Docker/Kubernetes, where an internal name may not resolve from where the producer runs), or the broker process being down. This overlaps with the advertised-listener problem covered in LEADER_NOT_AVAILABLE — if the bootstrap connection succeeds but subsequent metadata points at an unreachable address, that page covers the specific fix.

Fix 2: confirm the topic's metadata resolves

Connectivity to the cluster can succeed while metadata for the specific topic still fails to resolve — a subtly different problem with the same symptom:

kafka-topics.sh --bootstrap-server localhost:9092 --describe --topic orders

# exists and healthy:
# Topic: orders  PartitionCount: 3  ReplicationFactor: 3
#     Partition: 0  Leader: 1  Replicas: 1,2,3  Isr: 1,2,3
#
# UnknownTopicOrPartitionException means it doesn't exist yet

If the topic doesn't exist and auto.create.topics.enable is off on the broker (the increasingly common production default), every send for it queues, waits the full delivery.timeout.ms, and expires. Create the topic explicitly rather than relying on auto-creation, especially in any environment where you've deliberately disabled it.

Fix 3: check for buffer saturation

If connectivity and metadata are both fine, the producer may simply be generating records faster than the network and brokers can absorb them. Records queue in buffer.memory, and if that queue backs up long enough, batches at the front expire before ever being sent:

# check producer-side metrics for saturation signals
# (via JMX, or your client library's metrics() call)
buffer-available-bytes    # trending toward 0 = queue backing up
record-queue-time-avg     # rising = records waiting longer before being sent
request-latency-avg       # rising = brokers or network are the bottleneck

# mitigate:
buffer.memory=67108864     # default 32MB — raise if bursts are legitimate and temporary
batch.size=32768           # default 16KB — larger batches can improve throughput
linger.ms=20                # small delay to let batches fill, reducing request count

Raising buffer.memory buys time during a temporary burst, but if your sustained production rate genuinely exceeds what the brokers can absorb, it only delays the same expiry — the real fix in that case is more partitions, more broker capacity, or slowing down the producer to match reality.

Fix 4: stop treating sends as fire-and-forget

The most consequential fix isn't about the timeout at all — it's about whether your application ever finds out this happened:

// ❌ fire-and-forget — an expired batch fails completely silently in your app
producer.send(record);

// ✅ callback — you actually learn about the failure and can act on it
producer.send(record, (metadata, exception) -> {
    if (exception != null) {
        log.error("Failed to send record to {}: {}", record.topic(), exception.getMessage());
        // e.g. write to a local retry queue, alert, or increment a metric
    }
});

// ✅ or synchronously, if throughput allows it
try {
    producer.send(record).get();
} catch (ExecutionException e) {
    // e.getCause() is the actual TimeoutException
}

Without a callback or a checked future, an expired batch produces one log line from the client library and nothing else — no exception surfaces in your application code, no metric increments, nothing an on-call engineer would notice. If you've been debugging "missing records" without seeing this exception anywhere in your own logs, this is very likely why: the client saw it, your application never did.

Don't just raise the timeout

It's tempting to make the symptom disappear by increasing delivery.timeout.ms. Resist that as a first move — it doesn't fix connectivity, missing topics, or insufficient capacity; it just makes your producer wait longer before reporting the same underlying problem, and delays your own detection of it correspondingly. Use it only as a deliberate, temporary buffer while the real cause above is being fixed.

Debugging checklist

• ✓ Check broker reachability with kafka-broker-api-versions.sh before anything else
• ✓ Confirm the topic exists and has a leader with kafka-topics.sh --describe
• ✓ If both are fine, check producer buffer metrics for saturation (buffer-available-bytes, record-queue-time-avg)
• ✓ Confirm your code actually inspects send results — add a callback if it doesn't
• ✓ Remember delivery.timeout.ms must be ≥ request.timeout.ms + linger.ms
• ✓ Don't raise the timeout as a permanent fix — it hides the cause rather than resolving it