Consider you are having two or more instances of a the same service, in a microservices setup using java/spring/hibernate. To make this example a bit more clear let’s assume we have:
booksservice, which creates/updates booksbook-detailsservice which will process details of the bookstasksservice which will create some task for the created/updated books
What I’m trying to solve and find out the best way to handle this is related to the following scenarios:
- The same event (for example book 1 was updated) is emitted multiple times
- The
book 1 updatedupdated event is picked bybook-detailsservice which after its processing will emitbook 1 details updated event. Both those events will be handled intasksservice.
What tasks will eventually do is to do a check if exists and create or remove task for this subject (book 1 in this case). This can be a complex operation that cannot be an atomic db transaction.
Given that multiple instances subscribe to the same topic, the same event can be processed at the same time by multiple instances causing integrity constraint errors (due to unique indexes).
How this can be properly handled? The ideal would be having book 1 processing by instance #1 but that not blocking book 2 being processed by instance #2. While same book 1 events being queued up for the same instance.
Answer
You can use distributed locks and maps in your case. Hazelcast for example.
locks – https://docs.hazelcast.org/docs/3.0/manual/html/ch02s07.html.
maps – https://docs.hazelcast.org/docs/3.0/manual/html/ch02.html#Map
- Get lock by event(or event id..) – to prevent execution on differents instances
- Check map ( as example
IMap<BookEvent, boolean wasProcessed>), if this event was already processed on another instance – return - Process your event
- Put event and status to map
- Unlock