Celery的 chain和 group组合会失败
我在尝试使用Celery的 chain与 group实现以下目标:
- 将一组任务按批次执行;
- 每个任务都可以定义自己的队列和任务ID。
我在测试代码中加入 time.sleep,以假设任务的运行时间。
结果与我的预期不同。
日志显示两组任务同时被接收并完成。
我怀疑问题可能是由si引起的,因此我添加了一个中间任务,但结果仍然不符合预期。
下面是我的代码。
chain(group([task_one.s(i).set(task_id=f"task-20260318-{i}",link=task_two.s(f"task-20260318-{i}")) for i in range(3)]),barrier.s(),group([task_one.si(i).set(task_id=f"task-20260318-{i}",link=task_two.s(f"task-20260318-{i}")) for i in [4,5,6,7]]),)()
@app.task
def task_one(i):
time.sleep(10)
return f"The {i}th task was executed successfully."
@app.task
def task_two(result, task_id):
return f"Execution completed:{result}, task_id:{task_id}"
@app.task
def barrier(x):
return f"Midpoint:{x}"
下面是Celery的配置。
app.conf.update(
{
"broker_url": f"redis://:{settings.REDIS_PASSWORD}@{settings.REDIS_HOST}:{settings.REDIS_PORT}/{settings.REDIS_DB}",
"timezone": "Asia/Shanghai",
"enable_utc": False,
"accept_content": ["json"],
"task_serializer": "json",
"task_time_limit": 60 * 60 * 4,
"beat_dburi": f"postgresql+psycopg2://{settings.POSTGRES_USER}:{settings.POSTGRES_PASSWORD}@{settings.POSTGRES_HOST}:{settings.POSTGRES_PORT}/{settings.POSTGRES_DB}?options=-csearch_path=public",
"result_backend": f"db+postgresql://{settings.POSTGRES_USER}:{settings.POSTGRES_PASSWORD}@{settings.POSTGRES_HOST}:{settings.POSTGRES_PORT}/{settings.POSTGRES_DB}?options=-csearch_path=public",
"result_extended": True,
"result_serializer": "json",
"result_expires": 60 * 60 * 24 * 7,
"worker_force_execv": True,
"worker_max_tasks_per_child": 200,
"beat_tz_aware": False,
"broker_connection_retry_on_startup": True,
"event_queue_prefix": "dbmonitor:celery:event:",
"broker_transport_options": {
"visibility_timeout": 3600,
"fanout_prefix": True,
"fanout_patterns": True,
"global_keyprefix": "dbmonitor:celery:broker:",
},
"result_backend_transport_options": {
"global_keyprefix": "dbmonitor:celery:result:",
},
"task_default_queue": settings.DEFAULT_QUEUE,
}
)
我再次尝试后,现在按我想要的方式执行,但出现了新的错误:
- 在执行第五个任务时出现了任务重复的问题。
- 出现错误:任务celery.chord_unlock[570bfc1a-c522-40ad-aae9-deb983bc0848] 重试:1.0s后重试。
下面是一个新的代码尝试。
chain(
group(
[task_one.si(1).set(queue=QUEUE, link=task_two.s(1).set(queue=QUEUE)), task_one.si(2).set(queue=QUEUE, link=task_two.s(2).set(queue=QUEUE))]
),
group(
[task_one.si(3).set(queue=QUEUE, link=task_two.s(3).set(queue=QUEUE)), task_one.si(4).set(queue=QUEUE, link=task_two.s(4).set(queue=QUEUE))]
),
group(
[task_one.si(5).set(queue=QUEUE, link=task_two.s(5).set(queue=QUEUE)), task_one.si(6).set(queue=QUEUE, link=task_two.s(6).set(queue=QUEUE))]
),
group(
[task_one.si(7).set(queue=QUEUE, link=task_two.s(7).set(queue=QUEUE)), task_one.si(8).set(queue=QUEUE, link=task_two.s(8).set(queue=QUEUE))]
),
barrier.s().set(queue=QUEUE),
)()
以下是日志输出。
Mar 20 11:08:56 dba-monitor-server celery[2461608]: [2026-03-20 11:08:56,891: INFO/MainProcess] Task dbmonitor.operators.tasks.task_one[f89e0187-161c-4e4a-be04-1859fbdbd9ba] received
Mar 20 11:08:56 dba-monitor-server celery[2461608]: [2026-03-20 11:08:56,896: INFO/MainProcess] Task dbmonitor.operators.tasks.task_one[adefcbaf-b2dd-4924-bd0d-d603bfa791e3] received
Mar 20 11:09:06 dba-monitor-server celery[2461608]: [2026-03-20 11:09:06,907: INFO/MainProcess] Task dbmonitor.operators.tasks.task_two[7d443052-68a7-4735-a496-f429b6d454ce] received
Mar 20 11:09:06 dba-monitor-server celery[2461608]: [2026-03-20 11:09:06,912: INFO/MainProcess] Task dbmonitor.operators.tasks.task_two[589b300f-10ab-4ed6-9b09-67a34a3162f5] received
Mar 20 11:09:06 dba-monitor-server celery[2461608]: [2026-03-20 11:09:06,914: INFO/ForkPoolWorker-31] Task dbmonitor.operators.tasks.task_one[f89e0187-161c-4e4a-be04-1859fbdbd9ba] succeeded in 10.021168902516365s: 'The 1th task was executed successfully.'
Mar 20 11:09:06 dba-monitor-server celery[2461608]: [2026-03-20 11:09:06,916: INFO/ForkPoolWorker-1] Task dbmonitor.operators.tasks.task_two[7d443052-68a7-4735-a496-f429b6d454ce] succeeded in 0.007270157337188721s: 'Execution completed:The 1th task was executed successfully., task_id:1'
Mar 20 11:09:06 dba-monitor-server celery[2461608]: [2026-03-20 11:09:06,917: INFO/ForkPoolWorker-32] Task dbmonitor.operators.tasks.task_one[adefcbaf-b2dd-4924-bd0d-d603bfa791e3] succeeded in 10.019537463784218s: 'The 2th task was executed successfully.'
Mar 20 11:09:06 dba-monitor-server celery[2461608]: [2026-03-20 11:09:06,921: INFO/ForkPoolWorker-33] Task dbmonitor.operators.tasks.task_two[589b300f-10ab-4ed6-9b09-67a34a3162f5] succeeded in 0.007834792137145996s: 'Execution completed:The 2th task was executed successfully., task_id:2'
Mar 20 11:09:06 dba-monitor-server celery[2461608]: [2026-03-20 11:09:06,983: INFO/MainProcess] Task dbmonitor.operators.tasks.task_one[f55f8763-42bb-47f7-8f61-a201363535a4] received
Mar 20 11:09:06 dba-monitor-server celery[2461608]: [2026-03-20 11:09:06,986: INFO/MainProcess] Task dbmonitor.operators.tasks.task_one[3df4f269-7752-4488-a161-c45c8eb6d88c] received
Mar 20 11:09:16 dba-monitor-server celery[2461608]: [2026-03-20 11:09:16,991: INFO/MainProcess] Task dbmonitor.operators.tasks.task_two[a135a308-dcb2-4402-819a-dbc8a9fd005b] received
Mar 20 11:09:16 dba-monitor-server celery[2461608]: [2026-03-20 11:09:16,997: INFO/MainProcess] Task dbmonitor.operators.tasks.task_one[a749cc89-b0ec-4bed-adad-95247efd8f68] received
Mar 20 11:09:17 dba-monitor-server celery[2461608]: [2026-03-20 11:09:17,000: INFO/MainProcess] Task dbmonitor.operators.tasks.task_two[8541256c-697d-4bbc-86e3-16f8c52235f9] received
Mar 20 11:09:17 dba-monitor-server celery[2461608]: [2026-03-20 11:09:17,000: INFO/ForkPoolWorker-1] Task dbmonitor.operators.tasks.task_two[a135a308-dcb2-4402-819a-dbc8a9fd005b] succeeded in 0.007699117064476013s: 'Execution completed:The 3th task was executed successfully., task_id:3'
Mar 20 11:09:17 dba-monitor-server celery[2461608]: [2026-03-20 11:09:17,004: INFO/MainProcess] Task dbmonitor.operators.tasks.task_one[91384df5-04be-4328-bbea-0e1ce9827284] received
Mar 20 11:09:17 dba-monitor-server celery[2461608]: [2026-03-20 11:09:17,005: INFO/ForkPoolWorker-31] Task dbmonitor.operators.tasks.task_one[f55f8763-42bb-47f7-8f61-a201363535a4] succeeded in 10.020828753709793s: 'The 3th task was executed successfully.'
Mar 20 11:09:17 dba-monitor-server celery[2461608]: [2026-03-20 11:09:17,008: INFO/ForkPoolWorker-1] Task dbmonitor.operators.tasks.task_two[8541256c-697d-4bbc-86e3-16f8c52235f9] succeeded in 0.005624815821647644s: 'Execution completed:The 4th task was executed successfully., task_id:4'
Mar 20 11:09:17 dba-monitor-server celery[2461608]: [2026-03-20 11:09:17,008: INFO/MainProcess] Task dbmonitor.operators.tasks.task_one[a749cc89-b0ec-4bed-adad-95247efd8f68] received
Mar 20 11:09:17 dba-monitor-server celery[2461608]: [2026-03-20 11:09:17,011: INFO/ForkPoolWorker-32] Task dbmonitor.operators.tasks.task_one[3df4f269-7752-4488-a161-c45c8eb6d88c] succeeded in 10.022311344742775s: 'The 4th task was executed successfully.'
Mar 20 11:09:17 dba-monitor-server celery[2461608]: [2026-03-20 11:09:17,013: INFO/MainProcess] Task dbmonitor.operators.tasks.task_one[91384df5-04be-4328-bbea-0e1ce9827284] received
Mar 20 11:09:27 dba-monitor-server celery[2461608]: [2026-03-20 11:09:27,013: INFO/MainProcess] Task dbmonitor.operators.tasks.task_two[7d0d90de-a2a7-4690-9ae8-8be5ed79d03e] received
Mar 20 11:09:27 dba-monitor-server celery[2461608]: [2026-03-20 11:09:27,020: INFO/ForkPoolWorker-33] Task dbmonitor.operators.tasks.task_one[a749cc89-b0ec-4bed-adad-95247efd8f68] succeeded in 10.021471485495567s: 'The 5th task was executed successfully.'
Mar 20 11:09:27 dba-monitor-server celery[2461608]: [2026-03-20 11:09:27,020: INFO/MainProcess] Task dbmonitor.operators.tasks.task_two[e3fe0aa5-b577-4e3e-8c12-5c3784fd5272] received
Mar 20 11:09:27 dba-monitor-server celery[2461608]: [2026-03-20 11:09:27,022: INFO/ForkPoolWorker-2] Task dbmonitor.operators.tasks.task_two[7d0d90de-a2a7-4690-9ae8-8be5ed79d03e] succeeded in 0.007368505001068115s: 'Execution completed:The 5th task was executed successfully., task_id:5'
Mar 20 11:09:27 dba-monitor-server celery[2461608]: [2026-03-20 11:09:27,024: INFO/MainProcess] Task dbmonitor.operators.tasks.task_two[66654ae7-c8bd-4476-83d7-dd342ed51af8] received
Mar 20 11:09:27 dba-monitor-server celery[2461608]: [2026-03-20 11:09:27,026: INFO/ForkPoolWorker-31] Task dbmonitor.operators.tasks.task_one[91384df5-04be-4328-bbea-0e1ce9827284] succeeded in 10.019291758537292s: 'The 6th task was executed successfully.'
Mar 20 11:09:27 dba-monitor-server celery[2461608]: [2026-03-20 11:09:27,027: INFO/ForkPoolWorker-33] Task dbmonitor.operators.tasks.task_two[e3fe0aa5-b577-4e3e-8c12-5c3784fd5272] succeeded in 0.0054441094398498535s: 'Execution completed:The 6th task was executed successfully., task_id:6'
Mar 20 11:09:27 dba-monitor-server celery[2461608]: [2026-03-20 11:09:27,028: INFO/MainProcess] Task dbmonitor.operators.tasks.task_two[31ce8511-5825-44f6-9272-4d9591d0380d] received
Mar 20 11:09:27 dba-monitor-server celery[2461608]: [2026-03-20 11:09:27,030: INFO/ForkPoolWorker-1] Task dbmonitor.operators.tasks.task_one[a749cc89-b0ec-4bed-adad-95247efd8f68] succeeded in 10.019818812608719s: 'The 5th task was executed successfully.'
Mar 20 11:09:27 dba-monitor-server celery[2461608]: [2026-03-20 11:09:27,036: INFO/ForkPoolWorker-2] Task dbmonitor.operators.tasks.task_two[66654ae7-c8bd-4476-83d7-dd342ed51af8] succeeded in 0.01110781729221344s: 'Execution completed:The 5th task was executed successfully., task_id:5'
Mar 20 11:09:27 dba-monitor-server celery[2461608]: [2026-03-20 11:09:27,037: INFO/ForkPoolWorker-31] Task dbmonitor.operators.tasks.task_two[31ce8511-5825-44f6-9272-4d9591d0380d] succeeded in 0.007356494665145874s: 'Execution completed:The 6th task was executed successfully., task_id:6'
Mar 20 11:09:27 dba-monitor-server celery[2461608]: [2026-03-20 11:09:27,037: INFO/ForkPoolWorker-32] Task dbmonitor.operators.tasks.task_one[91384df5-04be-4328-bbea-0e1ce9827284] succeeded in 10.022301405668259s: 'The 6th task was executed successfully.'
Mar 20 11:09:27 dba-monitor-server celery[2461608]: [2026-03-20 11:09:27,081: INFO/MainProcess] Task celery.chord_unlock[811136a9-88b3-407e-bbe2-88d9354f6594] received
Mar 20 11:09:27 dba-monitor-server celery[2461608]: [2026-03-20 11:09:27,083: INFO/MainProcess] Task dbmonitor.operators.tasks.task_one[e273a293-2508-4d4a-ad9d-96161fd7c690] received
Mar 20 11:09:27 dba-monitor-server celery[2461608]: [2026-03-20 11:09:27,086: INFO/MainProcess] Task dbmonitor.operators.tasks.task_one[f6bcfb11-1575-492a-8892-20a6f6006b40] received
Mar 20 11:09:27 dba-monitor-server celery[2461608]: [2026-03-20 11:09:27,094: INFO/MainProcess] Task celery.chord_unlock[570bfc1a-c522-40ad-aae9-deb983bc0848] received
Mar 20 11:09:27 dba-monitor-server celery[2461608]: [2026-03-20 11:09:27,097: INFO/MainProcess] Task dbmonitor.operators.tasks.task_one[e273a293-2508-4d4a-ad9d-96161fd7c690] received
Mar 20 11:09:27 dba-monitor-server celery[2461608]: [2026-03-20 11:09:27,100: INFO/MainProcess] Task dbmonitor.operators.tasks.task_one[f6bcfb11-1575-492a-8892-20a6f6006b40] received
Mar 20 11:09:28 dba-monitor-server celery[2461608]: [2026-03-20 11:09:28,078: INFO/ForkPoolWorker-2] Task celery.chord_unlock[811136a9-88b3-407e-bbe2-88d9354f6594] retry: Retry in 1.0s
Mar 20 11:09:28 dba-monitor-server celery[2461608]: [2026-03-20 11:09:28,079: INFO/MainProcess] Task celery.chord_unlock[811136a9-88b3-407e-bbe2-88d9354f6594] received
.....
Mar 20 11:09:36 dba-monitor-server celery[2461608]: [2026-03-20 11:09:36,132: INFO/ForkPoolWorker-34] Task celery.chord_unlock[570bfc1a-c522-40ad-aae9-deb983bc0848] retry: Retry in 1.0s
Mar 20 11:09:36 dba-monitor-server celery[2461608]: [2026-03-20 11:09:36,133: INFO/MainProcess] Task celery.chord_unlock[570bfc1a-c522-40ad-aae9-deb983bc0848] received
Mar 20 11:09:37 dba-monitor-server celery[2461608]: [2026-03-20 11:09:37,099: INFO/MainProcess] Task dbmonitor.operators.tasks.task_two[cd8a6721-f7d9-4891-a936-6a091300a9b3] received
Mar 20 11:09:37 dba-monitor-server celery[2461608]: [2026-03-20 11:09:37,102: INFO/MainProcess] Task dbmonitor.operators.tasks.task_two[43777bfd-2430-4e7a-93ef-2d09e93c6eed] received
Mar 20 11:09:37 dba-monitor-server celery[2461608]: [2026-03-20 11:09:37,105: INFO/ForkPoolWorker-31] Task dbmonitor.operators.tasks.task_one[e273a293-2508-4d4a-ad9d-96161fd7c690] succeeded in 10.020209103822708s: 'The 7th task was executed successfully.'
Mar 20 11:09:37 dba-monitor-server celery[2461608]: [2026-03-20 11:09:37,107: INFO/ForkPoolWorker-32] Task dbmonitor.operators.tasks.task_one[f6bcfb11-1575-492a-8892-20a6f6006b40] succeeded in 10.01945075392723s: 'The 8th task was executed successfully.'
Mar 20 11:09:37 dba-monitor-server celery[2461608]: [2026-03-20 11:09:37,108: INFO/ForkPoolWorker-2] Task dbmonitor.operators.tasks.task_two[cd8a6721-f7d9-4891-a936-6a091300a9b3] succeeded in 0.007255181670188904s: 'Execution completed:The 7th task was executed successfully., task_id:7'
Mar 20 11:09:37 dba-monitor-server celery[2461608]: [2026-03-20 11:09:37,111: INFO/ForkPoolWorker-34] Task dbmonitor.operators.tasks.task_two[43777bfd-2430-4e7a-93ef-2d09e93c6eed] succeeded in 0.006953775882720947s: 'Execution completed:The 8th task was executed successfully., task_id:8'
Mar 20 11:09:37 dba-monitor-server celery[2461608]: [2026-03-20 11:09:37,112: INFO/MainProcess] Task dbmonitor.operators.tasks.task_two[85105574-1746-45a2-b0df-e892f7674956] received
Mar 20 11:09:37 dba-monitor-server celery[2461608]: [2026-03-20 11:09:37,115: INFO/MainProcess] Task dbmonitor.operators.tasks.task_two[8da8a70a-993d-4b5f-b980-afee0284af99] received
Mar 20 11:09:37 dba-monitor-server celery[2461608]: [2026-03-20 11:09:37,116: INFO/ForkPoolWorker-1] Task dbmonitor.operators.tasks.task_one[e273a293-2508-4d4a-ad9d-96161fd7c690] succeeded in 10.01807950437069s: 'The 7th task was executed successfully.'
Mar 20 11:09:37 dba-monitor-server celery[2461608]: [2026-03-20 11:09:37,119: INFO/ForkPoolWorker-31] Task dbmonitor.operators.tasks.task_two[85105574-1746-45a2-b0df-e892f7674956] succeeded in 0.006020337343215942s: 'Execution completed:The 7th task was executed successfully., task_id:7'
Mar 20 11:09:37 dba-monitor-server celery[2461608]: [2026-03-20 11:09:37,120: INFO/ForkPoolWorker-33] Task dbmonitor.operators.tasks.task_one[f6bcfb11-1575-492a-8892-20a6f6006b40] succeeded in 10.01877723634243s: 'The 8th task was executed successfully.'
Mar 20 11:09:37 dba-monitor-server celery[2461608]: [2026-03-20 11:09:37,123: INFO/ForkPoolWorker-32] Task dbmonitor.operators.tasks.task_two[8da8a70a-993d-4b5f-b980-afee0284af99] succeeded in 0.0059270113706588745s: 'Execution completed:The 8th task was executed successfully., task_id:8'
Mar 20 11:09:37 dba-monitor-server celery[2461608]: [2026-03-20 11:09:37,133: INFO/ForkPoolWorker-31] Task celery.chord_unlock[811136a9-88b3-407e-bbe2-88d9354f6594] succeeded in 0.006766557693481445s: None
Mar 20 11:09:37 dba-monitor-server celery[2461608]: [2026-03-20 11:09:37,135: INFO/MainProcess] Task dbmonitor.operators.tasks.barrier[6e1499ff-d58c-4d14-8411-d6164234496e] received
Mar 20 11:09:37 dba-monitor-server celery[2461608]: [2026-03-20 11:09:37,137: INFO/ForkPoolWorker-32] Task celery.chord_unlock[570bfc1a-c522-40ad-aae9-deb983bc0848] succeeded in 0.00584140419960022s: None
Mar 20 11:09:37 dba-monitor-server celery[2461608]: [2026-03-20 11:09:37,138: INFO/MainProcess] Task dbmonitor.operators.tasks.barrier[6e1499ff-d58c-4d14-8411-d6164234496e] received
Mar 20 11:09:37 dba-monitor-server celery[2461608]: [2026-03-20 11:09:37,141: INFO/ForkPoolWorker-31] Task dbmonitor.operators.tasks.barrier[6e1499ff-d58c-4d14-8411-d6164234496e] succeeded in 0.005473405122756958s: 'Midpoint:[\'The 7th task was executed successfully.\', \'The 8th task was executed successfully.\']'
Mar 20 11:09:37 dba-monitor-server celery[2461608]: [2026-03-20 11:09:37,143: WARNING/ForkPoolWorker-32] Failed operation _store_result. Retrying 2 more times.
Mar 20 11:09:37 dba-monitor-server celery[2461608]: Traceback (most recent call last):
Mar 20 11:09:37 dba-monitor-server celery[2461608]: File "/Data/Projects/MonitorTools/dbmonitor/.venv/lib/python3.9/site-packages/sqlalchemy/engine/base.py", line 1967, in _exec_single_context
Mar 20 11:09:37 dba-monitor-server celery[2461608]: self.dialect.do_execute(
Mar 20 11:09:37 dba-monitor-server celery[2461608]: File "/Data/Projects/MonitorTools/dbmonitor/.venv/lib/python3.9/site-packages/sqlalchemy/engine/default.py", line 951, in do_execute
Mar 20 11:09:37 dba-monitor-server celery[2461608]: cursor.execute(statement, parameters)
Mar 20 11:09:37 dba-monitor-server celery[2461608]: psycopg2.errors.UniqueViolation: duplicate key value violates unique constraint "celery_taskmeta_task_id_key"
Mar 20 11:09:37 dba-monitor-server celery[2461608]: DETAIL: Key (task_id)=(6e1499ff-d58c-4d14-8411-d6164234496e) already exists.
Mar 20 11:09:37 dba-monitor-server celery[2461608]: The above exception was the direct cause of the following exception:
Mar 20 11:09:37 dba-monitor-server celery[2461608]: Traceback (most recent call last):
Mar 20 11:09:37 dba-monitor-server celery[2461608]: File "/Data/Projects/MonitorTools/dbmonitor/.venv/lib/python3.9/site-packages/celery/backends/database/__init__.py", line 47, in _inner
Mar 20 11:09:37 dba-monitor-server celery[2461608]: return fun(*args, **kwargs)
Mar 20 11:09:37 dba-monitor-server celery[2461608]: File "/Data/Projects/MonitorTools/dbmonitor/.venv/lib/python3.9/site-packages/celery/backends/database/__init__.py", line 135, in _store_result
Mar 20 11:09:37 dba-monitor-server celery[2461608]: session.flush()
Mar 20 11:09:37 dba-monitor-server celery[2461608]: File "/Data/Projects/MonitorTools/dbmonitor/.venv/lib/python3.9/site-packages/sqlalchemy/orm/session.py", line 4331, in flush
Mar 20 11:09:37 dba-monitor-server celery[2461608]: self._flush(objects)
Mar 20 11:09:37 dba-monitor-server celery[2461608]: File "/Data/Projects/MonitorTools/dbmonitor/.venv/lib/python3.9/site-packages/sqlalchemy/orm/session.py", line 4467, in _flush
Mar 20 11:09:37 dba-monitor-server celery[2461608]: transaction.rollback(_capture_exception=True)
Mar 20 11:09:37 dba-monitor-server celery[2461608]: File "/Data/Projects/MonitorTools/dbmonitor/.venv/lib/python3.9/site-packages/sqlalchemy/util/langhelpers.py", line 224, in __exit__
Mar 20 11:09:37 dba-monitor-server celery[2461608]: raise exc_value.with_traceback(exc_tb)
Mar 20 11:09:37 dba-monitor-server celery[2461608]: File "/Data/Projects/MonitorTools/dbmonitor/.venv/lib/python3.9/site-packages/sqlalchemy/orm/session.py", line 4427, in _flush
Mar 20 11:09:37 dba-monitor-server celery[2461608]: flush_context.execute()
Mar 20 11:09:37 dba-monitor-server celery[2461608]: File "/Data/Projects/MonitorTools/dbmonitor/.venv/lib/python3.9/site-packages/sqlalchemy/orm/unitofwork.py", line 466, in execute
Mar 20 11:09:37 dba-monitor-server celery[2461608]: rec.execute(self)
Mar 20 11:09:37 dba-monitor-server celery[2461608]: File "/Data/Projects/MonitorTools/dbmonitor/.venv/lib/python3.9/site-packages/sqlalchemy/orm/unitofwork.py", line 642, in execute
Mar 20 11:09:37 dba-monitor-server celery[2461608]: util.preloaded.orm_persistence.save_obj(
Mar 20 11:09:37 dba-monitor-server celery[2461608]: File "/Data/Projects/MonitorTools/dbmonitor/.venv/lib/python3.9/site-packages/sqlalchemy/orm/persistence.py", line 93, in save_obj
Mar 20 11:09:37 dba-monitor-server celery[2461608]: _emit_insert_statements(
Mar 20 11:09:37 dba-monitor-server celery[2461608]: File "/Data/Projects/MonitorTools/dbmonitor/.venv/lib/python3.9/site-packages/sqlalchemy/orm/persistence.py", line 1233, in _emit_insert_statements
Mar 20 11:09:37 dba-monitor-server celery[2461608]: result = connection.execute(
Mar 20 11:09:37 dba-monitor-server celery[2461608]: File "/Data/Projects/MonitorTools/dbmonitor/.venv/lib/python3.9/site-packages/sqlalchemy/engine/base.py", line 1419, in execute
Mar 20 11:09:37 dba-monitor-server celery[2461608]: return meth(
Mar 20 11:09:37 dba-monitor-server celery[2461608]: File "/Data/Projects/MonitorTools/dbmonitor/.venv/lib/python3.9/site-packages/sqlalchemy/sql/elements.py", line 526, in _execute_on_connection
Mar 20 11:09:37 dba-monitor-server celery[2461608]: return connection._execute_clauseelement(
Mar 20 11:09:37 dba-monitor-server celery[2461608]: File "/Data/Projects/MonitorTools/dbmonitor/.venv/lib/python3.9/site-packages/sqlalchemy/engine/base.py", line 1641, in _execute_clauseelement
Mar 20 11:09:37 dba-monitor-server celery[2461608]: ret = self._execute_context(
Mar 20 11:09:37 dba-monitor-server celery[2461608]: File "/Data/Projects/MonitorTools/dbmonitor/.venv/lib/python3.9/site-packages/sqlalchemy/engine/base.py", line 1846, in _execute_context
Mar 20 11:09:37 dba-monitor-server celery[2461608]: return self._exec_single_context(
Mar 20 11:09:37 dba-monitor-server celery[2461608]: File "/Data/Projects/MonitorTools/dbmonitor/.venv/lib/python3.9/site-packages/sqlalchemy/engine/base.py", line 1986, in _exec_single_context
Mar 20 11:09:37 dba-monitor-server celery[2461608]: self._handle_dbapi_exception(
Mar 20 11:09:37 dba-monitor-server celery[2461608]: File "/Data/Projects/MonitorTools/dbmonitor/.venv/lib/python3.9/site-packages/sqlalchemy/engine/base.py", line 2355, in _handle_dbapi_exception
Mar 20 11:09:37 dba-monitor-server celery[2461608]: raise sqlalchemy_exception.with_traceback(exc_info[2]) from e
Mar 20 11:09:37 dba-monitor-server celery[2461608]: File "/Data/Projects/MonitorTools/dbmonitor/.venv/lib/python3.9/site-packages/sqlalchemy/engine/base.py", line 1967, in _exec_single_context
Mar 20 11:09:37 dba-monitor-server celery[2461608]: self.dialect.do_execute(
Mar 20 11:09:37 dba-monitor-server celery[2461608]: File "/Data/Projects/MonitorTools/dbmonitor/.venv/lib/python3.9/site-packages/sqlalchemy/engine/default.py", line 951, in do_execute
Mar 20 11:09:37 dba-monitor-server celery[2461608]: cursor.execute(statement, parameters)
Mar 20 11:09:37 dba-monitor-server celery[2461608]: sqlalchemy.exc.IntegrityError: (psycopg2.errors.UniqueViolation) duplicate key value violates unique constraint "celery_taskmeta_task_id_key"
Mar 20 11:09:37 dba-monitor-server celery[2461608]: DETAIL: Key (task_id)=(6e1499ff-d58c-4d14-8411-d6164234496e) already exists.
Mar 20 11:09:37 dba-monitor-server celery[2461608]: [SQL: INSERT INTO celery_taskmeta (id, task_id, status, result, date_done, traceback, name, args, kwargs, worker, retries, queue) VALUES (nextval('task_id_sequence'), %(task_id)s, %(status)s, %(result)s, %(date_done)s, %(traceback)s, %(name)s, %(args)s, %(kwargs)s, %(worker)s, %(retries)s, %(queue)s) RETURNING celery_taskmeta.id]
Mar 20 11:09:37 dba-monitor-server celery[2461608]: [parameters: {'task_id': '6e1499ff-d58c-4d14-8411-d6164234496e', 'status': 'PENDING', 'result': None, 'date_done': datetime.datetime(2026, 3, 20, 2, 56, 32, 257418, tzinfo=datetime.timezone.utc), 'traceback': None, 'name': None, 'args': None, 'kwargs': None, 'worker': None, 'retries': None, 'queue': None}]
Mar 20 11:09:37 dba-monitor-server celery[2461608]: (Background on this error at: https://sqlalche.me/e/20/gkpj)
Mar 20 11:09:37 dba-monitor-server celery[2461608]: [2026-03-20 11:09:37,147: INFO/ForkPoolWorker-32] Task dbmonitor.operators.tasks.barrier[6e1499ff-d58c-4d14-8411-d6164234496e] succeeded in 0.007541939616203308s: 'Midpoint:[\'The 7th task was executed successfully.\', \'The 8th task was executed successfully.\']'
解决方案
Celery本质上就是围绕批量执行,因此你的第1 点要求已经满足。同样地,每个Celery任务也可以发送到你想要的任意队列,因此第2 点的要求也已满足。
站内所有文章版权归属LeftHeroAI导航站,无授权禁止任何主体转载、抄袭、复制内容,亦不得私自架设镜像站点。一经侵权,本站将通过法律途径追责。