ื•ื•ื™ ืงืึทืคืงืึท ืื™ื– ื’ืขื•ื•ืืจืŸ ืคืึทืงื˜

ื•ื•ื™ ืงืึทืคืงืึท ืื™ื– ื’ืขื•ื•ืืจืŸ ืคืึทืงื˜

ื”ื™ื™ ื”ืื‘ืจ!

ืื™ืš ืึทืจื‘ืขื˜ ืื•ื™ืฃ ื“ื™ ื˜ื™ื ืงืึธืคืฃ ืžืึทื ืฉืึทืคึฟื˜, ื•ื•ืึธืก ืื™ื– ื“ืขื•ื•ืขืœืึธืคึผื™ื ื’ ื–ื™ื™ืŸ ืื™ื™ื’ืขื ืข ืึธื ื–ืึธื’ ืฆืขื ื˜ืขืจ. ืื™ืš ืžืขืจืกื˜ื ืก ืึทื ื˜ื•ื•ื™ืงืœืขืŸ ืื™ืŸ ื“ื–ืฉืึทื•ื•ืึท ื ื™ืฆืŸ ืกืคึผืจื™ื ื’ ืฉื˜ื™ื•ื•ืœ ืื•ืŸ ืกืึธืœื•ื•ืข ืคืึทืจืฉื™ื“ืŸ ื˜ืขื›ื ื™ืฉ ืคึผืจืึธื‘ืœืขืžืก ื•ื•ืึธืก ืฉื˜ื™ื™ืขืŸ ืื™ืŸ ืึท ืคึผืจื•ื™ืขืงื˜.

ืจื•ื‘ึฟ ืคื•ืŸ ืื•ื ื“ื–ืขืจ ืžื™ืงืจืึธืกืขืจื•ื•ื™ืกืขืก ื™ื‘ืขืจื’ืขื‘ืŸ ืžื™ื˜ ื™ืขื“ืขืจ ืื ื“ืขืจืข ื™ื™ืกื™ื ื’ืงืจืึทื ืึทืกืœื™ ื“ื•ืจืš ืึท ืึธื ื–ืึธื’ ืžืขืงืœืขืจ. ื‘ื™ื– ืึทื”ืขืจ, ืžื™ืจ ื’ืขื•ื•ื™ื™ื ื˜ IBM MQ ื•ื•ื™ ืึท ืžืขืงืœืขืจ, ื•ื•ืึธืก ืงืขืŸ ื ื™ื˜ ืžืขืจ ืงืึธืคึผืข ืžื™ื˜ ื“ื™ ืžืึทืกืข, ืึธื‘ืขืจ ืื™ืŸ ื“ืขืจ ื–ืขืœื‘ื™ืงืขืจ ืฆื™ื™ึทื˜ ื”ืื˜ ื”ื•ื™ืš ืขืงืกืคึผืจืขืก ื’ืขืจืึทื ื˜ื™ื–.

ื•ื•ื™ ืึท ืคืึทืจื‘ื™ื™ึทื˜, ืžื™ืจ ื–ืขื ืขืŸ ื’ืขืคึฟื™ื ื˜ Apache Kafka, ื•ื•ืึธืก ื”ืื˜ ืึท ื”ื•ื™ืš ืกืงื™ื™ืœื™ื ื’ ืคึผืึธื˜ืขื ืฆื™ืขืœ, ืึธื‘ืขืจ, ืœื™ื™ื“ืขืจ, ืจื™ืงื•ื•ื™ื™ืขืจื– ืึท ื›ึผืžืขื˜ ื™ื—ื™ื“ ืฆื•ื’ืึทื ื’ ืฆื• ืงืึทื ืคื™ื’ื™ืขืจื™ื™ืฉืึทืŸ ืคึฟืึทืจ ืคืึทืจืฉื™ื“ืขื ืข ืกื™ื ืขืจื™ืึธื•ื–. ืื™ืŸ ืึทื“ื™ืฉืึทืŸ, ื“ื™ ืขืงืกืคึผืจืขืก ืžืขืงืึทื ื™ื–ืึทื ืื™ืŸ ืžื™ื ื“ืกื˜ืขืจ ืึทืžืึธืœ ื•ื•ืึธืก ืึทืจื‘ืขื˜ ืื™ืŸ Kafka ื“ื•ืจืš ืคืขืœื™ืงื™ื™ึทื˜ ื”ืื˜ ื ื™ืฉื˜ ืœืึธื–ืŸ ื“ื™ ืคืืจืœืื ื’ื˜ ืงืึธื ืกื™ืกื˜ืขื ืกื™ ืžื“ืจื’ื” ืคื•ืŸ ื“ื™ ืงืขืกื˜ืœ. ื“ืขืจื ืึธืš, ืื™ืš ื•ื•ืขืœ ื˜ื™ื™ืœืŸ ืื•ื ื“ื–ืขืจ ื“ืขืจืคืึทืจื•ื ื’ ืื™ืŸ ืงืึทืคืงืึท ืงืึทื ืคื™ื’ื™ืขืจื™ื™ืฉืึทืŸ, ืื™ืŸ ื‘ืึทื–ื•ื ื“ืขืจ, ืื™ืš ื•ื•ืขืœ ื–ืึธื’ืŸ ืื™ืจ ื•ื•ื™ ืฆื• ืงืึทื ืคื™ื’ื™ืขืจ ืื•ืŸ ืœืขื‘ืŸ ืžื™ื˜ ืคึผื•ื ืงื˜ ืึทืžืึธืœ ืขืงืกืคึผืจืขืก.

ื’ืขืจืึทื ื˜ื™ื“ ืขืงืกืคึผืจืขืก ืื•ืŸ ืžืขืจ

ื“ื™ ืกืขื˜ื˜ื™ื ื’ืก ื“ื™ืกืงืึทืกื˜ ืื•ื ื˜ืŸ ื•ื•ืขื˜ ื”ืขืœืคืŸ ืคืึทืจืžื™ื™ึทื“ืŸ ืึท ื ื•ืžืขืจ ืคื•ืŸ ืคึผืจืึธื‘ืœืขืžืก ืžื™ื˜ ื“ื™ ืคืขืœื™ืงื™ื™ึทื˜ ืคึฟืึทืจื‘ื™ื ื“ื•ื ื’ ืกืขื˜ื˜ื™ื ื’ืก. ืื‘ืขืจ ืขืจืฉื˜ืขืจ ืื™ืš ื•ื•ืึธืœื˜ ื•ื•ื™ ืฆื• ื‘ืึทืฆืึธืœืŸ ื•ืคืžืขืจืงื–ืึทืžืงื™ื™ึทื˜ ืฆื• ืื™ื™ืŸ ืคึผืึทืจืึทืžืขื˜ืขืจ ื•ื•ืึธืก ื•ื•ืขื˜ ืคืึทืกื™ืœืึทื˜ื™ื™ื˜ ืึท ืžืขื’ืœืขืš ื“ื™ื‘ืึทื’.

ื“ืืก ื•ื•ืขื˜ ื”ืขืœืคืŸ client.id ืคึฟืึทืจ ืคึผืจืึธื“ื•ืฆื™ืจืขืจ ืื•ืŸ ืงืึธื ืกื•ืžืขืจ. ืื™ืŸ ืขืจืฉื˜ืขืจ ื‘ืœื™ืง, ืื™ืจ ืงืขื ืขืŸ ื ื•ืฆืŸ ื“ื™ ืึทืคึผืœืึทืงื™ื™ืฉืึทืŸ ื ืึธืžืขืŸ ื•ื•ื™ ื“ื™ ื•ื•ืขืจื˜, ืื•ืŸ ืื™ืŸ ืจื•ื‘ึฟ ืงืึทืกืขืก ื“ืึธืก ื•ื•ืขื˜ ืึทืจื‘ืขื˜ืŸ. ื›ืึธื˜ืฉ ื“ื™ ืกื™ื˜ื•ืึทืฆื™ืข ื•ื•ืขืŸ ืึท ืึทืคึผืœืึทืงื™ื™ืฉืึทืŸ ื ื™ืฆื˜ ืขื˜ืœืขื›ืข ืงืึธื ืกื•ืžืขืจืก ืื•ืŸ ืื™ืจ ื’ืขื‘ืŸ ื–ื™ื™ ื“ื™ ื–ืขืœื‘ืข client.id, ืจืขื–ื•ืœื˜ืึทื˜ืŸ ืื™ืŸ ื“ื™ ืคืืœื’ืขื ื“ืข ื•ื•ืืจืขื ื•ื ื’:

org.apache.kafka.common.utils.AppInfoParser โ€” Error registering AppInfo mbean javax.management.InstanceAlreadyExistsException: kafka.consumer:type=app-info,id=kafka.test-0

ืื•ื™ื‘ ืื™ืจ ื•ื•ื™ืœืŸ ืฆื• ื ื•ืฆืŸ JMX ืื™ืŸ ืึท ืึทืคึผืœืึทืงื™ื™ืฉืึทืŸ ืžื™ื˜ Kafka, ื“ืึธืก ืงืขืŸ ื–ื™ื™ืŸ ืึท ืคึผืจืึธื‘ืœืขื. ืคึฟืึทืจ ื“ืขื ืคืึทืœ, ืขืก ืื™ื– ื‘ืขืกื˜ืขืจ ืฆื• ื ื•ืฆืŸ ืึท ืงืึธืžื‘ื™ื ืึทืฆื™ืข ืคื•ืŸ โ€‹โ€‹โ€‹โ€‹ื“ื™ ืึทืคึผืœืึทืงื™ื™ืฉืึทืŸ ื ืึธืžืขืŸ ืื•ืŸ, ืคึฟืึทืจ ื‘ื™ื™ึทืฉืคึผื™ืœ, ื“ื™ ื˜ืขืžืข ื ืึธืžืขืŸ ื•ื•ื™ ื“ื™ client.id ื•ื•ืขืจื˜. ื“ืขืจ ืจืขื–ื•ืœื˜ืึทื˜ ืคื•ืŸ ืื•ื ื“ื–ืขืจ ืงืึทื ืคื™ื’ื™ืขืจื™ื™ืฉืึทืŸ ืงืขื ืขืŸ ื–ื™ื™ืŸ ื’ืขื–ืขืŸ ืื™ืŸ ื“ื™ ื‘ืึทืคึฟืขืœ ืจืขื–ื•ืœื˜ืึทื˜ ืงืึทืคืงืึท-ืงืึทื ืกื•ืžืขืจ-ื’ืจื•ืคึผืขืก ืคื•ืŸ ื™ื•ื˜ื™ืœืึทื˜ื™ื– ืคื•ืŸ ืงืึธื ืคืœื•ืขื ื˜:

ื•ื•ื™ ืงืึทืคืงืึท ืื™ื– ื’ืขื•ื•ืืจืŸ ืคืึทืงื˜

ืื™ืฆื˜ ืœืึธื–ืŸ ืก ืงื•ืง ืื™ืŸ ื“ื™ ืกืฆืขื ืึทืจ ืคึฟืึทืจ ื’ืขืจืึทื ื˜ื™ื“ ืึธื ื–ืึธื’ ืขืงืกืคึผืจืขืก. Kafka ืคึผืจืึธื“ื•ืฆื™ืจืขืจ ื”ืื˜ ืึท ืคึผืึทืจืึทืžืขื˜ืขืจ ืึทืงืก, ื•ื•ืึธืก ืึทืœืึทื•ื– ืื™ืจ ืฆื• ืงืึทื ืคื™ื’ื™ืขืจ ื ืึธืš ื•ื•ื™ ืคื™ืœืข ื™ืงื ืึทืœื™ื“ื–ืฉื– ื“ืขืจ ืงื ื•ื™ืœ ืคื™ืจืขืจ ื“ืึทืจืฃ ืฆื• ื‘ืึทื˜ืจืึทื›ื˜ืŸ ื“ื™ ืึธื ื–ืึธื’ ื”ืฆืœื—ื” ื’ืขืฉืจื™ื‘ืŸ. ื“ืขืจ ืคึผืึทืจืึทืžืขื˜ืขืจ ืงืขืŸ ื ืขืžืขืŸ ื“ื™ ืคืืœื’ืขื ื“ืข ื•ื•ืึทืœื•ืขืก:

  • 0 - ื‘ืึทืฉื˜ืขื˜ื™ืงืŸ ื•ื•ืขื˜ ื ื™ื˜ ื–ื™ื™ืŸ ื‘ืึทื˜ืจืึทื›ื˜.
  • 1 ืื™ื– ื“ื™ ืคืขืœื™ืงื™ื™ึทื˜ ืคึผืึทืจืึทืžืขื˜ืขืจ, ื‘ืœื•ื™ื– 1 ืจืขืคึผืœื™ืงืข ืื™ื– ืคืืจืœืื ื’ื˜ ืฆื• ื‘ืึทืฉื˜ืขื˜ื™ืงืŸ.
  • -1 - ื‘ืึทืฉื˜ืขื˜ื™ืงืŸ ืคื•ืŸ ืึทืœืข ืกื™ื ื’ืงืจืึทื ื™ื™ื–ื“ ืจืขืคึผืœืึทืงืึทื– ืื™ื– ืคืืจืœืื ื’ื˜ (ืงื ื•ื™ืœ ืกืขื˜ืึทืคึผ min.insync.replicas).

ืคึฟื•ืŸ ื“ื™ ืœื™ืกื˜ืขื“ ื•ื•ืึทืœื•ืขืก ืขืก ืื™ื– ืงืœืึธืจ ืึทื– ืึทืงืก ื’ืœื™ื™ึทืš ืฆื• -1 ื’ื™ื˜ ื“ื™ ืกื˜ืจืึธื ื’ืขืกื˜ ื’ืึทืจืึทื ื˜ื™ืจืŸ ืึทื– ื“ืขืจ ืึธื ื–ืึธื’ ื•ื•ืขื˜ ื ื™ืฉื˜ ื–ื™ื™ืŸ ืคืึทืจืคืึทืœืŸ.

ื•ื•ื™ ืžื™ืจ ืึทืœืข ื•ื•ื™ืกืŸ, ื“ื™ืกื˜ืจื™ื‘ื™ื•ื˜ื™ื“ ืกื™ืกื˜ืขืžืขืŸ ื–ืขื ืขืŸ ืึทื ืจื™ืœื™ื™ืึทื‘ืึทืœ. ืฆื• ื‘ืึทืฉื™ืฆืŸ ืงืขื’ืŸ ื˜ืจืึทื ื–ืฉืึทื ื˜ ื—ืกืจื•ื ื•ืช, Kafka ืคึผืจืึธื“ื•ืกืขืจ ื’ื™ื˜ ื“ื™ ืึธืคึผืฆื™ืข ืจื™ื˜ืจื™ืขืก, ื•ื•ืึธืก ืึทืœืึทื•ื– ืื™ืจ ืฆื• ืฉื˜ืขืœืŸ ื“ื™ ื ื•ืžืขืจ ืคื•ืŸ ืจืขืกืขื ื“ ืคืจื•ื•ื•ืŸ ื™ืŸ ืขืงืกืคึผืจืขืก.ื˜ื™ืžืขืึธื•ื˜.ืžืก. ื–ื™ื ื˜ ื“ื™ ืจืขื˜ืจื™ืขืก ืคึผืึทืจืึทืžืขื˜ืขืจ ื”ืื˜ ืึท ืคืขืœื™ืงื™ื™ึทื˜ ื•ื•ืขืจื˜ ืคื•ืŸ Integer.MAX_VALUE (2147483647), ื“ื™ ื ื•ืžืขืจ ืคื•ืŸ ืึธื ื–ืึธื’ ืจื™ื˜ืจื™ืขืก ืงืขื ืขืŸ ื–ื™ื™ืŸ ืึทื“ื–ืฉืึทืกื˜ื™ื“ ื“ื•ืจืš ื˜ืฉืึทื ื’ื™ื ื’ ื‘ืœื•ื™ื– ื“ืขืœื™ื•ื•ืขืจื™.ื˜ื™ืžืขืึธื•ื˜.ืžืก.

ืžื™ืจ ื–ืขื ืขืŸ ืžืึธื•ื•ื™ื ื’ ืฆื• ืคึผื•ื ืงื˜ ืึทืžืึธืœ ืขืงืกืคึผืจืขืก

ื“ื™ ืœื™ืกื˜ืขื“ ืกืขื˜ื˜ื™ื ื’ืก ืœืึธื–ืŸ ืื•ื ื“ื–ืขืจ ืคึผืจืึธื“ื•ืฆื™ืจืขืจ ืฆื• ื‘ืึทืคืจื™ื™ึทืขืŸ ืึทืจื˜ื™ืงืœืขืŸ ืžื™ื˜ ืึท ื”ื•ื™ืš ื’ืึทืจืึทื ื˜ื™ืจืŸ. ื–ืืœ ืก ืื™ืฆื˜ ืจืขื“ืŸ ื•ื•ืขื’ืŸ ื•ื•ื™ ืฆื• ืขื ืฉื•ืจ ืึทื– ื‘ืœื•ื™ื– ืื™ื™ืŸ ืงืึธืคึผื™ืข ืคื•ืŸ โ€‹โ€‹ืึท ืึธื ื–ืึธื’ ืื™ื– ื’ืขืฉืจื™ื‘ืŸ ืฆื• ืึท ืงืึทืคืงืึท ื˜ืขืžืข? ืื™ืŸ ื“ื™ ืกื™ืžืคึผืœืึทืกื˜ ืคืึทืœ, ืฆื• ื˜ืึธืŸ ื“ืึธืก, ืื™ืจ ื“ืึทืจืคึฟืŸ ืฆื• ืฉื˜ืขืœืŸ ื“ืขื ืคึผืึทืจืึทืžืขื˜ืขืจ ืื•ื™ืฃ ืคึผืจืึธื“ื•ืกืขืจ ื’ืขื‘ืŸ.ื™ื“ืขืžืคึผืึธื˜ืขื ืกืข ืฆื• ืืžืช. ืื™ื“ืขืžืคึผืึธื˜ืขื ืกื™ ื’ืขืจืึทื ื˜ื™ื– ืึทื– ื‘ืœื•ื™ื– ืื™ื™ืŸ ืึธื ื–ืึธื’ ืื™ื– ื’ืขืฉืจื™ื‘ืŸ ืฆื• ืึท ืกืคึผืขืฆื™ืคื™ืฉ ืฆืขื˜ื™ื™ืœื•ื ื’ ืคื•ืŸ ืื™ื™ืŸ ื˜ืขืžืข. ื“ื™ ืคึผืจื™ื“ื™ืกืคึผืึทื–ื™ืฉืึทืŸ ืคึฟืึทืจ ื™ื ื™ื™ื‘ืึทืœื™ื ื’ ื™ื™ื“ืขืžืคึผืึธื˜ืขื ืกื™ ืื™ื– ื“ื™ ื•ื•ืึทืœื•ืขืก ืึทืงืงืก = ืึทืœืข, ืคึผืจื•ื‘ื™ืจืŸ ื•ื•ื™ื“ืขืจ > 0, max.in.flight.requests.per.connection โ‰ค 5. ืื•ื™ื‘ ื“ื™ ืคึผืึทืจืึทืžืขื˜ืขืจืก ื–ืขื ืขืŸ ื ื™ืฉื˜ ืกืคึผืขืกื™ืคื™ืขื“ ื“ื•ืจืš ื“ื™ ื“ืขื•ื•ืขืœืึธืคึผืขืจ, ื“ื™ ืื•ื™ื‘ืŸ ื•ื•ืึทืœื•ืขืก ื•ื•ืขื˜ ื–ื™ื™ืŸ ืื•ื™ื˜ืึธืžืึทื˜ื™ืฉ ื‘ืึทืฉื˜ื™ืžื˜.

ื•ื•ืขืŸ ืื™ื“ืขืžืคึผืึธื˜ืขื ืกื™ ืื™ื– ืงืึทื ืคื™ื’ื™ืขืจื“, ืขืก ืื™ื– ื ื™ื™ื˜ื™ืง ืฆื• ืขื ืฉื•ืจ ืึทื– ื“ื™ ื–ืขืœื‘ืข ืึทืจื˜ื™ืงืœืขืŸ ืกื•ืฃ ืึทืจื•ื™ืฃ ืื™ืŸ ื“ืขืจ ื–ืขืœื‘ื™ืงืขืจ ืคึผืึทืจื˜ื™ืฉืึทื ื– ื™ืขื“ืขืจ ืžืึธืœ. ื“ืขื ืงืขื ืขืŸ ื•ื•ืขืจืŸ ื’ืขื˜ืืŸ ื“ื•ืจืš ื‘ืึทืฉื˜ืขื˜ื™ืงืŸ ื“ื™ ืคึผืึทืจื˜ื™ืึธื ืขืจ.ืงืœืึทืก ืฉืœื™ืกืœ ืื•ืŸ ืคึผืึทืจืึทืžืขื˜ืขืจ ืฆื• ืคึผืจืึธื“ื•ืกืขืจ. ื–ืืœ ืก ืึธื ื”ื™ื™ื‘ืŸ ืžื™ื˜ ื“ื™ ืฉืœื™ืกืœ. ืขืก ืžื•ื–ืŸ ื–ื™ื™ืŸ ื“ื™ ื–ืขืœื‘ืข ืคึฟืึทืจ ื™ืขื“ืขืจ ืกืึทื‘ืžื™ืฉืึทืŸ. ื“ืึธืก ืงืขื ืขืŸ ืœื™ื™ื›ื˜ ื–ื™ื™ืŸ ืึทื˜ืฉื™ื•ื•ื“ ื“ื•ืจืš ื ื™ืฆืŸ ืงื™ื™ืŸ ืคื•ืŸ ื“ื™ ื’ืขืฉืขืคื˜ ื™ื“ืก ืคื•ืŸ ื“ืขืจ ืึธืจื™ื’ื™ื ืขืœ ืคึผืึธืกื˜ืŸ. ื“ืขืจ ืคึผืึทืจืึทืžืขื˜ืขืจ partitioner.class ื”ืื˜ ืึท ืคืขืœื™ืงื™ื™ึทื˜ ื•ื•ืขืจื˜ - DefaultPartitioner. ืžื™ื˜ ื“ืขื ืคึผืึทืจื˜ื™ืฉืึทื ื™ื ื’ ืกื˜ืจืึทื˜ืขื’ื™ืข, ืžื™ืจ ื˜ืึธืŸ ื•ื•ื™ ื“ืึธืก:

  • ืื•ื™ื‘ ื“ื™ ืฆืขื˜ื™ื™ืœื•ื ื’ ืื™ื– ื‘ืคื™ืจื•ืฉ ืกืคึผืขืกื™ืคื™ืขื“ ื•ื•ืขืŸ ืฉื™ืงื˜ ื“ืขื ืึธื ื–ืึธื’, ืžื™ืจ ื ื•ืฆืŸ ืขืก.
  • ืื•ื™ื‘ ื“ื™ ืฆืขื˜ื™ื™ืœื•ื ื’ ืื™ื– ื ื™ืฉื˜ ืกืคึผืขืกื™ืคื™ืขื“, ืึธื‘ืขืจ ื“ืขืจ ืฉืœื™ืกืœ ืื™ื– ืกืคึผืขืกื™ืคื™ืขื“, ืกืขืœืขืงื˜ื™ืจืŸ ื“ืขื ืฆืขื˜ื™ื™ืœื•ื ื’ ื“ื•ืจืš ื“ื™ ื”ืึทืฉ ืคื•ืŸ ื“ื™ ืฉืœื™ืกืœ.
  • ืื•ื™ื‘ ื“ื™ ืฆืขื˜ื™ื™ืœื•ื ื’ ืื•ืŸ ืฉืœื™ืกืœ ื–ืขื ืขืŸ ื ื™ืฉื˜ ืกืคึผืขืกื™ืคื™ืขื“, ืื•ื™ืกืงืœื™ื™ึทื‘ืŸ ื“ื™ ืคึผืึทืจื˜ื™ืฉืึทื ื– ืื™ื™ื ืขืจ ื“ื•ืจืš ืื™ื™ื ืขืจ (ืงื™ื™ึทืœืขื›ื™ืง ืจืึธื‘ื™ืŸ).

ืื•ื™ืš, ื ื™ืฆืŸ ืึท ืฉืœื™ืกืœ ืื•ืŸ ืื™ื“ืขืžืคึผืึธื˜ืขื ื˜ ืฉื™ืงื˜ ืžื™ื˜ ืึท ืคึผืึทืจืึทืžืขื˜ืขืจ max.in.flight.requests.per.connection = 1 ื’ื™ื˜ ืื™ืจ ืกื˜ืจื™ืžืœื™ื™ื ื“ ืึธื ื–ืึธื’ ืคึผืจืึทืกืขืกื™ื ื’ ืื•ื™ืฃ ื“ื™ ืงืึธื ืกื•ืžืขืจ. ืขืก ืื™ื– ืื•ื™ืš ื•ื•ืขืจื˜ ืฆื• ื’ืขื“ืขื ืงืขืŸ ืึทื– ืื•ื™ื‘ ืึทืงืกืขืก ืงืึธื ื˜ืจืึธืœ ืื™ื– ืงืึทื ืคื™ื’ื™ืขืจื“ ืื•ื™ืฃ ื“ื™ื™ืŸ ืงื ื•ื™ืœ, ืื™ืจ ื•ื•ืขื˜ ื“ืึทืจืคึฟืŸ ืจืขื›ื˜ ืฆื• ื™ื™ื“ืขืžืคึผืึทื˜ืึทื ื˜ืœื™ ืฉืจื™ื™ึทื‘ืŸ ืฆื• ืึท ื˜ืขืžืข.

ืื•ื™ื‘ ืื™ืจ ืคึผืœื•ืฆืœื™ื ื’ ืคืขืœืŸ ื“ื™ ืงื™ื™ืคึผืึทื‘ื™ืœืึทื˜ื™ื– ืคื•ืŸ ื™ื™ื“ืขืžืคึผืึทื˜ืึทื ื˜ ืฉื™ืงื˜ ื“ื•ืจืš ืฉืœื™ืกืœ ืึธื“ืขืจ ื“ื™ ืœืึธื’ื™ืง ืื•ื™ืฃ ื“ื™ ืคึผืจืึธื“ื•ืฆื™ืจืขืจ ื–ื™ื™ึทื˜ ืจื™ืงื•ื•ื™ื™ืจื– ืžื™ื™ื ื˜ื™ื™ื ื™ื ื’ ื“ืึทื˜ืŸ ืงืึธื ืกื™ืกื˜ืขื ืกื™ ืฆื•ื•ื™ืฉืŸ ืคืึทืจืฉื™ื“ืขื ืข ืคึผืึทืจื˜ื™ืฉืึทื ื–, ื˜ืจืึทื ื–ืึทืงืฉืึทื ื– ื•ื•ืขื˜ ืงื•ืžืขืŸ ืฆื• ืจืึทื˜ืขื•ื•ืขืŸ. ืื™ืŸ ื“ืขืจืฆื•, ื ื™ืฆืŸ ืึท ืงื™ื™ื˜ ื˜ืจืึทื ืกืึทืงื˜ื™ืึธืŸ, ืื™ืจ ืงืขื ืขืŸ ืงืึทื ื“ื™ืฉืึทื ืึทืœื™ ืกื™ื ื’ืงืจืึทื ื™ื™ื– ืึท ืจืขืงืึธืจื“ ืื™ืŸ ืงืึทืคืงืึท, ืœืžืฉืœ, ืžื™ื˜ ืึท ืจืขืงืึธืจื“ ืื™ืŸ ื“ื™ ื“ืึทื˜ืึทื‘ื™ื™ืก. ืฆื• ื’ืขื‘ืŸ ื˜ืจืึทื ืกืึทืงื˜ื™ืึธื ืึทืœ ืฉื™ืงื˜ ืฆื• ื“ื™ ืคึผืจืึธื“ื•ืฆื™ืจืขืจ, ืขืก ืžื•ื–ืŸ ื–ื™ื™ืŸ ื™ื™ื“ืขืžืคึผืึทื˜ืึทื ื˜ ืื•ืŸ ืึทื“ื™ืฉื ืึทืœื™ ืฉื˜ืขืœืŸ ื˜ืจืึทื ืกืึทืงื˜ื™ืึธื ืึทืœ.ื™ื“. ืื•ื™ื‘ ื“ื™ื™ืŸ ืงืึทืคืงืึท ืงื ื•ื™ืœ ื”ืื˜ ืึทืงืกืขืก ืงืึธื ื˜ืจืึธืœ ืงืึทื ืคื™ื’ื™ืขืจื“, ืึท ื˜ืจืึทื ืกืึทืงื˜ื™ืึธื ืึทืœ ืจืขืงืึธืจื“, ื•ื•ื™ ืึท ื™ื™ื“ืขืžืคึผืึธื˜ืขื ื˜ ืจืขืงืึธืจื“, ื•ื•ืขื˜ ื“ืึทืจืคึฟืŸ ืฉืจื™ื™ึทื‘ืŸ ืคึผืขืจืžื™ืฉืึทื ื–, ื•ื•ืึธืก ืงืขื ืขืŸ ื–ื™ื™ืŸ ื“ืขืจืœื•ื™ื‘ื˜ ื“ื•ืจืš ืžืึทืกืงืข ื ื™ืฆืŸ ื“ื™ ื•ื•ืขืจื˜ ืกื˜ืึธืจื“ ืื™ืŸ transactional.id.

ืคืึธืจืžืึทืœื™, ืงื™ื™ืŸ ืฉื˜ืจื™ืงืœ, ืึทื–ืึท ื•ื•ื™ ื“ื™ ืึทืคึผืœืึทืงื™ื™ืฉืึทืŸ ื ืึธืžืขืŸ, ืงืขื ืขืŸ ื•ื•ืขืจืŸ ื’ืขื ื•ืฆื˜ ื•ื•ื™ ืึท ื˜ืจืึทื ืกืึทืงื˜ื™ืึธืŸ ื™ื“ืขื ื˜ื™ืคื™ืขืจ. ืึธื‘ืขืจ ืื•ื™ื‘ ืื™ืจ ืงืึทื˜ืขืจ ืขื˜ืœืขื›ืข ื™ื ืกื˜ืึทื ืกื™ื– ืคื•ืŸ ื“ืขืจ ื–ืขืœื‘ื™ืงืขืจ ืึทืคึผืœืึทืงื™ื™ืฉืึทืŸ ืžื™ื˜ ื“ืขืจ ื–ืขืœื‘ื™ืงืขืจ transactional.id, ื“ืขืจ ืขืจืฉื˜ืขืจ ืœืึธื ื˜ืฉื˜ ื‘ื™ื™ึทืฉืคึผื™ืœ ื•ื•ืขื˜ ื–ื™ื™ืŸ ืกื˜ืึทืคึผื˜ ืžื™ื˜ ืึท ื˜ืขื•ืช, ื•ื•ื™ื™ึทืœ Kafka ื•ื•ืขื˜ ื‘ืึทื˜ืจืึทื›ื˜ืŸ ืขืก ื•ื•ื™ ืึท ื–ืึทืžื‘ื™ ืคึผืจืึธืฆืขืก.

org.apache.kafka.common.errors.ProducerFencedException: Producer attempted an operation with an old epoch. Either there is a newer producer with the same transactionalId, or the producer's transaction has been expired by the broker.

ืฆื• ืกืึธืœื•ื•ืข ื“ืขื ืคึผืจืึธื‘ืœืขื, ืžื™ืจ ืœื™ื™ื’ืŸ ืึท ืกืึทืคื™ืงืก ืฆื• ื“ื™ ืึทืคึผืœืึทืงื™ื™ืฉืึทืŸ ื ืึธืžืขืŸ ืื™ืŸ ื“ื™ ืคืึธืจืขื ืคื•ืŸ ื“ื™ ื”ืึธืกื˜ื ืึทืžืข, ื•ื•ืึธืก ืžื™ืจ ื‘ืึทืงื•ืžืขืŸ ืคึฟื•ืŸ ืกื•ื•ื™ื•ื•ืข ื•ื•ืขืจื™ืึทื‘ืึทืœื–.

ื“ืขืจ ืคึผืจืึธื“ื•ืฆื™ืจืขืจ ืื™ื– ืงืึทื ืคื™ื’ื™ืขืจื“, ืึธื‘ืขืจ ื˜ืจืึทื ื–ืึทืงืฉืึทื ื– ืื•ื™ืฃ Kafka ืงืึธื ื˜ืจืึธืœื™ืจืŸ ื‘ืœื•ื™ื– ื“ื™ ืคืึทืจื ืขื ืคื•ืŸ ื“ืขื ืึธื ื–ืึธื’. ืจืขื’ืึทืจื“ืœืขืกืก ืคื•ืŸ ื“ื™ ื˜ืจืึทื ืกืึทืงื˜ื™ืึธืŸ ืกื˜ืึทื˜ื•ืก, ื“ืขืจ ืึธื ื–ืึธื’ ืžื™ื“ ื’ื™ื™ื˜ ืฆื• ื“ื™ ื˜ืขืžืข, ืึธื‘ืขืจ ื”ืื˜ ื ืึธืš ืกื™ืกื˜ืขื ืึทื˜ืจื™ื‘ื™ื•ืฅ.

ืฆื• ืคืึทืจืžื™ื™ึทื“ืŸ ืึทื–ืึท ืึทืจื˜ื™ืงืœืขืŸ ืฆื• ืœื™ื™ืขื ืขืŸ ื“ื•ืจืš ื“ื™ ืงืึธื ืกื•ืžืขืจ ืคืจื™ืขืจ, ืขืก ื“ืึทืจืฃ ืฆื• ืฉื˜ืขืœืŸ ื“ืขื ืคึผืึทืจืึทืžืขื˜ืขืจ isolation.level ืฆื• ืœื™ื™ืขื ืขืŸ_ืงืึทืžืžื™ื˜ื˜ืขื“ ื•ื•ืขืจื˜. ืึทื–ืึท ืึท ืงืึธื ืกื•ืžืขืจ ื•ื•ืขื˜ ืงืขื ืขืŸ ืฆื• ืœื™ื™ืขื ืขืŸ ื ื™ื˜-ื˜ืจืึทื ื–ืึทืงืฉืึทื ืึทืœ ืึทืจื˜ื™ืงืœืขืŸ ื•ื•ื™ ืคืจื™ืขืจ, ืื•ืŸ ื˜ืจืึทื ืกืึทืงื˜ื™ืึธื ืึทืœ ืึทืจื˜ื™ืงืœืขืŸ ื‘ืœื•ื™ื– ื ืึธืš ืึท ื™ื‘ืขืจื’ืขื‘ืŸ.
ืื•ื™ื‘ ืื™ืจ ื”ืึธื˜ ื‘ืึทืฉื˜ื™ืžื˜ ืึทืœืข ื“ื™ ืกืขื˜ื˜ื™ื ื’ืก ืœื™ืกื˜ืขื“ ืคืจื™ืขืจ, ืื™ืจ ื”ืึธื˜ ืงืึทื ืคื™ื’ื™ืขืจื“ ืคึผื•ื ืงื˜ ืึทืžืึธืœ ืขืงืกืคึผืจืขืก. ืžืึทื–ืœ - ื˜ืึธื•ื•!

ืื‘ืขืจ ืขืก ืื™ื– ื ืึธืš ืื™ื™ืŸ ื ื•ืึทื ืก. Transactional.id, ื•ื•ืึธืก ืžื™ืจ ืงืึทื ืคื™ื’ื™ืขืจื“ ืื•ื™ื‘ืŸ, ืื™ื– ืคืืงื˜ื™ืฉ ื“ื™ ื˜ืจืึทื ืกืึทืงื˜ื™ืึธืŸ ืคึผืจืขืคื™ืงืก. ืื•ื™ืฃ ื“ื™ ื˜ืจืึทื ืกืึทืงื˜ื™ืึธืŸ ืคืึทืจื•ื•ืึทืœื˜ืขืจ, ืึท ืกื™ืงื•ื•ืึทื ืก ื ื•ืžืขืจ ืื™ื– ืžื•ืกื™ืฃ ืฆื• ืขืก. ื“ืขืจ ื‘ืืงื•ืžืขืŸ ืื™ื“ืขื ื˜ื™ืคื™ืืจ ืื™ื– ืืจื•ื™ืก ืฆื• transactional.id.expiration.ms, ื•ื•ืึธืก ืื™ื– ืงืึทื ืคื™ื’ื™ืขืจื“ ืื•ื™ืฃ ืึท ืงืึทืคืงืึท ืงื ื•ื™ืœ ืื•ืŸ ื”ืื˜ ืึท ืคืขืœื™ืงื™ื™ึทื˜ ื•ื•ืขืจื˜ ืคื•ืŸ "7 ื˜ืขื’". ืื•ื™ื‘ ืื™ืŸ ื“ืขื ืฆื™ื™ื˜ ื“ื™ ืึทืคึผืœืึทืงื™ื™ืฉืึทืŸ ื”ืื˜ ื ื™ืฉื˜ ื‘ืืงื•ืžืขืŸ ืงื™ื™ืŸ ืึทืจื˜ื™ืงืœืขืŸ, ื•ื•ืขืŸ ืื™ืจ ืคึผืจื•ื‘ื™ืจืŸ ื“ื™ ื•ื•ื™ื™ึทื˜ืขืจ ื˜ืจืึทื ืกืึทืงื˜ื™ืึธื ืึทืœ ืฉื™ืงืŸ ืื™ืจ ื•ื•ืขื˜ ื‘ืึทืงื•ืžืขืŸ InvalidPidMappingException. ื“ืขืจ ื˜ืจืึทื ืกืึทืงื˜ื™ืึธืŸ ืงืึธืึธืจื“ื™ื ืึทื˜ืึธืจ ื•ื•ืขื˜ ื“ืขืžืึธืœื˜ ืึทืจื•ื™ืกื’ืขื‘ืŸ ืึท ื ื™ื™ึท ืกื™ืงื•ื•ืึทื ืก ื ื•ืžืขืจ ืคึฟืึทืจ ื“ืขืจ ื•ื•ื™ื™ึทื˜ืขืจ ื˜ืจืึทื ืกืึทืงื˜ื™ืึธืŸ. ืึธื‘ืขืจ, ื“ืขืจ ืึธื ื–ืึธื’ ืงืขืŸ ื–ื™ื™ืŸ ืคืึทืจืคืึทืœืŸ ืื•ื™ื‘ ื“ื™ InvalidPidMappingException ืื™ื– ื ื™ืฉื˜ ื›ืึทื ื“ืึทืœื“ ืจื™ื›ื˜ื™ืง.

ืึทื ืฉื˜ืึธื˜ ื˜ืึธื•ื˜ืึทืœื–

ื•ื•ื™ ืื™ืจ ืงืขื ืขืŸ ื–ืขืŸ, ืขืก ืื™ื– ื ื™ืฉื˜ ื’ืขื ื•ื’ ืฆื• ื ืึธืจ ืฉื™ืงืŸ ืึทืจื˜ื™ืงืœืขืŸ ืฆื• Kafka. ืื™ืจ ื“ืึทืจืคึฟืŸ ืฆื• ืงืœื™ื™ึทื‘ืŸ ืึท ืงืึธืžื‘ื™ื ืึทืฆื™ืข ืคื•ืŸ โ€‹โ€‹ืคึผืึทืจืึทืžืขื˜ืขืจืก ืื•ืŸ ื–ื™ื™ืŸ ืฆื•ื’ืขื’ืจื™ื™ื˜ ืฆื• ืžืึทื›ืŸ ืฉื ืขืœ ืขื ื“ืขืจื•ื ื’ืขืŸ. ืื™ืŸ ื“ืขื ืึทืจื˜ื™ืงืœ, ืื™ืš ื’ืขืคืจื•ื•ื•ื˜ ืฆื• ื•ื•ื™ื™ึทื–ืŸ ืื™ืŸ ื“ืขื˜ืึทืœ ื“ื™ ืคึผื•ื ืงื˜ ืึทืžืึธืœ ืขืงืกืคึผืจืขืก ืกืขื˜ืึทืคึผ ืื•ืŸ ื“ื™ืกืงืจื™ื™ื‘ื“ ืขื˜ืœืขื›ืข ืคืจืื‘ืœืขืžืขืŸ ืžื™ื˜ ื“ื™ client.id ืื•ืŸ transactional.id ืงืึทื ืคื™ื’ื™ืขืจื™ื™ืฉืึทื ื– ื•ื•ืึธืก ืžื™ืจ ื’ืขืคึผืœืึธื ื˜ืขืจื˜. ื•ื ื˜ืขืจ ืื™ื– ืึท ืงื™ืฆืขืจ ืคื•ืŸ ื“ื™ ืคึผืจืึธื“ื•ืฆื™ืจืขืจ ืื•ืŸ ืงืึธื ืกื•ืžืขืจ ืกืขื˜ื˜ื™ื ื’ืก.

ืคึผืจืึธื“ื•ืฆื™ืจืขืจ:

  1. ืึทืงืก = ืึทืœืข
  2. ืจื™ื˜ืจื™ื•ื–> 0
  3. ืขื ืึทื‘ืœืข.ื™ื“ืขืžืคึผืึธื˜ืขื ืกืข = ืืžืช
  4. max.in.flight.requests.per.connection โ‰ค 5 (1 ืคึฟืึทืจ ืึธืจื“ืขืจืœื™ ืฉื™ืงื˜)
  5. transactional.id = ${ืึทืคึผืคึผืœื™ืงืึทื˜ื™ืึธืŸ ื ืึธืžืขืŸ}-${ื”ืึธืกื˜ื ืึทืžืข}

ืงืึธื ืกื•ืžืขืจ:

  1. isolation.level = read_committed

ืฆื• ืžื™ื ืึทืžื™ื™ื– ืขืจืจืึธืจืก ืื™ืŸ ืฆื•ืงื•ื ืคึฟื˜ ืึทืคึผืœืึทืงื™ื™ืฉืึทื ื–, ืžื™ืจ ื’ืขืžืื›ื˜ ืื•ื ื“ื–ืขืจ ืื™ื™ื’ืขื ืข ืจืึทืคึผืขืจ ืื™ื‘ืขืจ ื“ื™ ืคืจื™ืœื™ื ื’ ืงืึทื ืคื™ื’ื™ืขืจื™ื™ืฉืึทืŸ, ื•ื•ื• ื•ื•ืึทืœื•ืขืก ืคึฟืึทืจ ืขื˜ืœืขื›ืข ืคื•ืŸ โ€‹โ€‹ื“ื™ ืœื™ืกื˜ืขื“ ืคึผืึทืจืึทืžืขื˜ืขืจืก ื–ืขื ืขืŸ ืฉื•ื™ืŸ ื‘ืึทืฉื˜ื™ืžื˜.

ื“ืึธ ื–ืขื ืขืŸ ืึท ืคึผืึธืจ ืคื•ืŸ ืžืึทื˜ืขืจื™ืึทืœืก ืคึฟืึทืจ ื–ื™ืš-ืœืขืจื ืขืŸ:

ืžืงื•ืจ: www.habr.com

ืœื™ื™ื’ืŸ ืึท ื‘ืึทืžืขืจืงื•ื ื’