Add Kafka listener idempotency

Contribution Overview

The Engineering Context

This contribution was made to the rajadilipkolli/spring-boot-microservices-series-v2open-source repository. The work involved understanding the existing codebase, identifying the underlying engineering problem, implementing a focused solution, and validating the change against the project's existing behaviour and testing standards.

Original Issue

Prevent duplicate Kafka messages from being processed multiple times.

Kafka consumers in the payment-service and inventory-service needed idempotency handling so duplicate message delivery would not result in the same business operation being processed repeatedly.

Engineering Problem

Understanding the Problem

Kafka provides at-least-once delivery semantics in many failure scenarios, which means consumers can receive duplicate messages. Without idempotent processing, duplicate delivery can cause payment or inventory operations to be executed multiple times and create inconsistent application state.

Solution

Implementation Approach

Implemented Kafka listener idempotency across the payment-service and inventory-service. Added processed-message tracking, duplicate detection using the message key, topic, and consumer group, and early-return handling for previously processed messages. Added database-level uniqueness protection and recorded messages only after successful processing. Integrated the mechanism into Kafka listeners and added integration coverage for inventory message processing.

Technologies

Engineering Stack