Hello
Today we tried an upgrade to the latest on-premise release, and install.sh has following output for failure
04:06:42 [INFO] sentry.plugins.github: apps-not-configured
Running migrations for default
Traceback (most recent call last):
File "/usr/local/bin/sentry", line 8, in <module>
sys.exit(main())
File "/usr/local/lib/python3.6/site-packages/sentry/runner/__init__.py", line 196, in main
func(**kwargs)
File "/usr/local/lib/python3.6/site-packages/click/core.py", line 829, in __call__
return self.main(*args, **kwargs)
File "/usr/local/lib/python3.6/site-packages/click/core.py", line 782, in main
rv = self.invoke(ctx)
File "/usr/local/lib/python3.6/site-packages/click/core.py", line 1259, in invoke
return _process_result(sub_ctx.command.invoke(sub_ctx))
File "/usr/local/lib/python3.6/site-packages/click/core.py", line 1066, in invoke
return ctx.invoke(self.callback, **ctx.params)
File "/usr/local/lib/python3.6/site-packages/click/core.py", line 610, in invoke
return callback(*args, **kwargs)
File "/usr/local/lib/python3.6/site-packages/click/decorators.py", line 21, in new_func
return f(get_current_context(), *args, **kwargs)
File "/usr/local/lib/python3.6/site-packages/sentry/runner/decorators.py", line 29, in inner
return ctx.invoke(f, *args, **kwargs)
File "/usr/local/lib/python3.6/site-packages/click/core.py", line 610, in invoke
return callback(*args, **kwargs)
File "/usr/local/lib/python3.6/site-packages/click/decorators.py", line 21, in new_func
return f(get_current_context(), *args, **kwargs)
File "/usr/local/lib/python3.6/site-packages/sentry/runner/commands/upgrade.py", line 100, in upgrade
_upgrade(not noinput, traceback, verbosity, not no_repair, with_nodestore)
File "/usr/local/lib/python3.6/site-packages/sentry/runner/commands/upgrade.py", line 56, in _upgrade
verbosity=verbosity,
File "/usr/local/lib/python3.6/site-packages/django/core/management/__init__.py", line 141, in call_command
return command.execute(*args, **defaults)
File "/usr/local/lib/python3.6/site-packages/django/core/management/base.py", line 335, in execute
output = self.handle(*args, **options)
File "/usr/local/lib/python3.6/site-packages/django/core/management/commands/migrate.py", line 82, in handle
executor.loader.check_consistent_history(connection)
File "/usr/local/lib/python3.6/site-packages/django/db/migrations/loader.py", line 292, in check_consistent_history
connection.alias,
django.db.migrations.exceptions.InconsistentMigrationHistory: Migration jira_ac.0001_initial is applied before its dependency sentry.0001_squashed_0200_release_indices on database 'default'.
An error occurred, caught SIGERR on line 12
Cleaning up...
After this upgrade failure, we have received massive error messages from job worker_1 as
ProgrammingError of column sentry_releaseprojectenvironment.adopted does not exist
worker_1 | 04:21:50 [ERROR] sentry_sdk.errors: Internal error in sentry_sdk
worker_1 | Traceback (most recent call last):
worker_1 | File "/usr/local/lib/python3.6/site-packages/sentry/db/postgres/decorators.py", line 91, in inner
worker_1 | return func(self, sql, *args, **kwargs)
worker_1 | File "/usr/local/lib/python3.6/site-packages/sentry/db/postgres/base.py", line 75, in execute
worker_1 | return self.cursor.execute(sql, clean_bad_params(params))
worker_1 | psycopg2.ProgrammingError: column sentry_releaseprojectenvironment.adopted does not exist
worker_1 | LINE 1: ...ntry_releaseprojectenvironment"."last_deploy_id", "sentry_re...
worker_1 | ^
worker_1 |
worker_1 |
worker_1 | During handling of the above exception, another exception occurred:
worker_1 |
worker_1 | Traceback (most recent call last):
worker_1 | File "/usr/local/lib/python3.6/site-packages/django/db/backends/utils.py", line 85, in _execute
worker_1 | return self.cursor.execute(sql, params)
worker_1 | File "/usr/local/lib/python3.6/site-packages/sentry/db/postgres/decorators.py", line 77, in inner
worker_1 | raise_the_exception(self.db, e)
worker_1 | File "/usr/local/lib/python3.6/site-packages/sentry/db/postgres/decorators.py", line 75, in inner
worker_1 | return func(self, *args, **kwargs)
worker_1 | File "/usr/local/lib/python3.6/site-packages/sentry/db/postgres/decorators.py", line 18, in inner
worker_1 | return func(self, *args, **kwargs)
worker_1 | File "/usr/local/lib/python3.6/site-packages/sentry/db/postgres/decorators.py", line 95, in inner
worker_1 | raise exc_info[0](msg).with_traceback(exc_info[2])
worker_1 | File "/usr/local/lib/python3.6/site-packages/sentry/db/postgres/decorators.py", line 91, in inner
worker_1 | return func(self, sql, *args, **kwargs)
worker_1 | File "/usr/local/lib/python3.6/site-packages/sentry/db/postgres/base.py", line 75, in execute
worker_1 | return self.cursor.execute(sql, clean_bad_params(params))
worker_1 | psycopg2.ProgrammingError: ProgrammingError('column sentry_releaseprojectenvironment.adopted does not exist\nLINE 1: ...ntry_releaseprojectenvironment"."last_deploy_id", "sentry_re...\n ^\n',)
worker_1 | SQL: SELECT "sentry_releaseprojectenvironment"."id", "sentry_releaseprojectenvironment"."release_id", "sentry_releaseprojectenvironment"."project_id", "sentry_releaseprojectenvironment"."environment_id", "sentry_releaseprojectenvironment"."new_issues_count", "sentry_releaseprojectenvironment"."first_seen", "sentry_releaseprojectenvironment"."last_seen", "sentry_releaseprojectenvironment"."last_deploy_id", "sentry_releaseprojectenvironment"."adopted", "sentry_releaseprojectenvironment"."unadopted" FROM "sentry_releaseprojectenvironment" WHERE ("sentry_releaseprojectenvironment"."environment_id" = %s AND "sentry_releaseprojectenvironment"."project_id" = %s AND "sentry_releaseprojectenvironment"."release_id" = %s)
worker_1 |
worker_1 | The above exception was the direct cause of the following exception:
worker_1 |
worker_1 | Traceback (most recent call last):
worker_1 | File "/usr/local/lib/python3.6/site-packages/celery/app/trace.py", line 412, in trace_task
worker_1 | R = retval = fun(*args, **kwargs)
worker_1 | File "/usr/local/lib/python3.6/site-packages/celery/app/trace.py", line 704, in __protected_call__
worker_1 | return self.run(*args, **kwargs)
worker_1 | File "/usr/local/lib/python3.6/site-packages/sentry_sdk/integrations/celery.py", line 197, in _inner
worker_1 | reraise(*exc_info)
worker_1 | File "/usr/local/lib/python3.6/site-packages/sentry_sdk/_compat.py", line 54, in reraise
worker_1 | raise value
worker_1 | File "/usr/local/lib/python3.6/site-packages/sentry_sdk/integrations/celery.py", line 192, in _inner
worker_1 | return f(*args, **kwargs)
worker_1 | File "/usr/local/lib/python3.6/site-packages/sentry/tasks/base.py", line 46, in _wrapped
worker_1 | result = func(*args, **kwargs)
worker_1 | File "/usr/local/lib/python3.6/site-packages/sentry/tasks/store.py", line 853, in save_event
worker_1 | _do_save_event(cache_key, data, start_time, event_id, project_id, **kwargs)
worker_1 | File "/usr/local/lib/python3.6/site-packages/sentry/tasks/store.py", line 767, in _do_save_event
worker_1 | project_id, assume_normalized=True, start_time=start_time, cache_key=cache_key
worker_1 | File "/usr/local/lib/python3.6/site-packages/sentry/utils/metrics.py", line 192, in inner
worker_1 | return f(*args, **kwargs)
worker_1 | File "/usr/local/lib/python3.6/site-packages/sentry/event_manager.py", line 449, in save
worker_1 | _get_or_create_release_associated_models(jobs, projects)
worker_1 | File "/usr/local/lib/python3.6/site-packages/sentry/utils/metrics.py", line 192, in inner
worker_1 | return f(*args, **kwargs)
worker_1 | File "/usr/local/lib/python3.6/site-packages/sentry/event_manager.py", line 751, in _get_or_create_release_associated_models
worker_1 | project=project, release=release, environment=environment, datetime=date
worker_1 | File "/usr/local/lib/python3.6/site-packages/sentry/models/releaseprojectenvironment.py", line 51, in get_or_create
worker_1 | release, project, environment, datetime, metrics_tags, **kwargs
worker_1 | File "/usr/local/lib/python3.6/site-packages/sentry/models/releaseprojectenvironment.py", line 65, in _get_or_create_impl
worker_1 | defaults={"first_seen": datetime, "last_seen": datetime},
worker_1 | File "/usr/local/lib/python3.6/site-packages/django/db/models/manager.py", line 82, in manager_method
worker_1 | return getattr(self.get_queryset(), name)(*args, **kwargs)
worker_1 | File "/usr/local/lib/python3.6/site-packages/django/db/models/query.py", line 487, in get_or_create
worker_1 | return self.get(**lookup), False
worker_1 | File "/usr/local/lib/python3.6/site-packages/django/db/models/query.py", line 397, in get
worker_1 | num = len(clone)
worker_1 | File "/usr/local/lib/python3.6/site-packages/django/db/models/query.py", line 254, in __len__
worker_1 | self._fetch_all()
worker_1 | File "/usr/local/lib/python3.6/site-packages/django/db/models/query.py", line 1182, in _fetch_all
worker_1 | self._result_cache = list(self._iterable_class(self))
worker_1 | File "/usr/local/lib/python3.6/site-packages/django/db/models/query.py", line 53, in __iter__
worker_1 | results = compiler.execute_sql(chunked_fetch=self.chunked_fetch, chunk_size=self.chunk_size)
worker_1 | File "/usr/local/lib/python3.6/site-packages/django/db/models/sql/compiler.py", line 1068, in execute_sql
worker_1 | cursor.execute(sql, params)
worker_1 | File "/usr/local/lib/python3.6/site-packages/sentry_sdk/integrations/django/__init__.py", line 500, in execute
worker_1 | return real_execute(self, sql, params)
worker_1 | File "/usr/local/lib/python3.6/site-packages/django/db/backends/utils.py", line 68, in execute
worker_1 | return self._execute_with_wrappers(sql, params, many=False, executor=self._execute)
worker_1 | File "/usr/local/lib/python3.6/site-packages/django/db/backends/utils.py", line 77, in _execute_with_wrappers
worker_1 | return executor(sql, params, many, context)
worker_1 | File "/usr/local/lib/python3.6/site-packages/django/db/backends/utils.py", line 85, in _execute
worker_1 | return self.cursor.execute(sql, params)
worker_1 | File "/usr/local/lib/python3.6/site-packages/django/db/utils.py", line 89, in __exit__
worker_1 | raise dj_exc_value.with_traceback(traceback) from exc_value
worker_1 | File "/usr/local/lib/python3.6/site-packages/django/db/backends/utils.py", line 85, in _execute
worker_1 | return self.cursor.execute(sql, params)
worker_1 | File "/usr/local/lib/python3.6/site-packages/sentry/db/postgres/decorators.py", line 77, in inner
worker_1 | raise_the_exception(self.db, e)
worker_1 | File "/usr/local/lib/python3.6/site-packages/sentry/db/postgres/decorators.py", line 75, in inner
worker_1 | return func(self, *args, **kwargs)
worker_1 | File "/usr/local/lib/python3.6/site-packages/sentry/db/postgres/decorators.py", line 18, in inner
worker_1 | return func(self, *args, **kwargs)
worker_1 | File "/usr/local/lib/python3.6/site-packages/sentry/db/postgres/decorators.py", line 95, in inner
worker_1 | raise exc_info[0](msg).with_traceback(exc_info[2])
worker_1 | File "/usr/local/lib/python3.6/site-packages/sentry/db/postgres/decorators.py", line 91, in inner
worker_1 | return func(self, sql, *args, **kwargs)
worker_1 | File "/usr/local/lib/python3.6/site-packages/sentry/db/postgres/base.py", line 75, in execute
worker_1 | return self.cursor.execute(sql, clean_bad_params(params))
worker_1 | django.db.utils.ProgrammingError: ProgrammingError('column sentry_releaseprojectenvironment.adopted does not exist\nLINE 1: ...ntry_releaseprojectenvironment"."last_deploy_id", "sentry_re...\n ^\n',)
worker_1 | SQL: SELECT "sentry_releaseprojectenvironment"."id", "sentry_releaseprojectenvironment"."release_id", "sentry_releaseprojectenvironment"."project_id", "sentry_releaseprojectenvironment"."environment_id", "sentry_releaseprojectenvironment"."new_issues_count", "sentry_releaseprojectenvironment"."first_seen", "sentry_releaseprojectenvironment"."last_seen", "sentry_releaseprojectenvironment"."last_deploy_id", "sentry_releaseprojectenvironment"."adopted", "sentry_releaseprojectenvironment"."unadopted" FROM "sentry_releaseprojectenvironment" WHERE ("sentry_releaseprojectenvironment"."environment_id" = %s AND "sentry_releaseprojectenvironment"."project_id" = %s AND "sentry_releaseprojectenvironment"."release_id" = %s)
worker_1 | 04:21:52 [ERROR] celery.app.trace: Task sentry.tasks.store.save_event[fedd9d43-80e5-4aa1-834a-cfe4f7640547] raised unexpected: ProgrammingError('ProgrammingError(\'column sentry_releaseprojectenvironment.adopted does not exist\\nLINE 1: ...ntry_releaseprojectenvironment"."last_deploy_id", "sentry_re...\\n ^\\n\',)\nSQL: SELECT "sentry_releaseprojectenvironment"."id", "sentry_releaseprojectenvironment"."release_id", "sentry_releaseprojectenvironment"."project_id", "sentry_releaseprojectenvironment"."environment_id", "sentry_releaseprojectenvironment"."new_issues_count", "sentry_releaseprojectenvironment"."first_seen", "sentry_releaseprojectenvironment"."last_seen", "sentry_releaseprojectenvironment"."last_deploy_id", "sentry_releaseprojectenvironment"."adopted", "sentry_releaseprojectenvironment"."unadopted" FROM "sentry_releaseprojectenvironment" WHERE ("sentry_releaseprojectenvironment"."environment_id" = %s AND "sentry_releaseprojectenvironment"."project_id" = %s AND "sentry_releaseprojectenvironment"."release_id" = %s)',) (data={'hostname': '4ba7ea085668', 'id': 'fedd9d43-80e5-4aa1-834a-cfe4f7640547', 'name': 'sentry.tasks.store.save_event', 'exc': 'ProgrammingError(\'ProgrammingError(\\\'column sentry_releaseprojectenvironment.adopted does not exist\\\\nLINE 1: ...ntry_releaseprojectenvironment"."last_deploy_id", "sentry_re...\\\\n ^\\\\n\\\',)\\nSQL: SELECT "sentry_releaseprojectenvironment"."id", "sentry_releaseprojectenvironment"."release_id", "sentry_releaseprojectenvironment"."project_id", "sentry_releaseprojectenvironment"."environment_id", "sentry_releaseprojectenvironment"."new_issues_count", "sentry_releaseprojectenvironment"."first_seen", "sentry_releaseprojectenvironment"."last_seen", "sentry_releaseprojectenvironment"."last_deploy_id", "sentry_releaseprojectenvironment"."adopted", "sentry_releaseprojectenvironment"."unadopted" FROM "sentry_releaseprojectenvironment" WHERE ("sentry_releaseprojectenvironment"."environment_id" = %s AND "sentry_releaseprojectenvironment"."project_id" = %s AND "sentry_releaseprojectenvironment"."release_id" = %s)\',)', 'traceback': 'Traceback (most recent call last):\n File "/usr/local/lib/python3.6/site-packages/sentry/db/postgres/decorators.py", line 91, in inner\n return func(self, sql, *args, **kwargs)\n File "/usr/local/lib/python3.6/site-packages/sentry/db/postgres/base.py", line 75, in execute\n return self.cursor.execute(sql, clean_bad_params(params))\npsycopg2.ProgrammingError: column sentry_releaseprojectenvironment.adopted does not exist\nLINE 1: ...ntry_releaseprojectenvironment"."last_deploy_id", "sentry_re...\n ^\n\n\nDuring handling of the above exception, another exception occurred:\n\nTraceback (most recent call last):\n File "/usr/local/lib/python3.6/site-packages/django/db/backends/utils.py", line 85, in _execute\n return self.cursor.execute(sql, params)\n File "/usr/local/lib/python3.6/site-packages/sentry/db/postgres/decorators.py", line 77, in inner\n raise_the_exception(self.db, e)\n File "/usr/local/lib/python3.6/site-packages/sentry/db/postgres/decorators.py", line 75, in inner\n return func(self, *args, **kwargs)\n File "/usr/local/lib/python3.6/site-packages/sentry/db/postgres/decorators.py", line 18, in inner\n return func(self, *args, **kwargs)\n File "/usr/local/lib/python3.6/site-packages/sentry/db/postgres/decorators.py", line 95, in inner\n raise exc_info[0](msg).with_traceback(exc_info[2])\n File "/usr/local/lib/python3.6/site-packages/sentry/db/postgres/decorators.py", line 91, in inner\n return func(self, sql, *args, **kwargs)\n File "/usr/local/lib/python3.6/site-packages/sentry/db/postgres/base.py", line 75, in execute\n return self.cursor.execute(sql, clean_bad_params(params))\npsycopg2.ProgrammingError: ProgrammingError(\'column sentry_releaseprojectenvironment.adopted does not exist\\nLINE 1: ...ntry_releaseprojectenvironment"."last_deploy_id", "sentry_re...\\n ^\\n\',)\nSQL: SELECT "sentry_releaseprojectenvironment"."id", "sentry_releaseprojectenvironment"."release_id", "sentry_releaseprojectenvironment"."project_id", "sentry_releaseprojectenvironment"."environment_id", "sentry_releaseprojectenvironment"."new_issues_count", "sentry_releaseprojectenvironment"."first_seen", "sentry_releaseprojectenvironment"."last_seen", "sentry_releaseprojectenvironment"."last_deploy_id", "sentry_releaseprojectenvironment"."adopted", "sentry_releaseprojectenvironment"."unadopted" FROM "sentry_releaseprojectenvironment" WHERE ("sentry_releaseprojectenvironment"."environment_id" = %s AND "sentry_releaseprojectenvironment"."project_id" = %s AND "sentry_releaseprojectenvironment"."release_id" = %s)\n\nThe above exception was the direct cause of the following exception:\n\nTraceback (most recent call last):\n File "/usr/local/lib/python3.6/site-packages/celery/app/trace.py", line 412, in trace_task\n R = retval = fun(*args, **kwargs)\n File "/usr/local/lib/python3.6/site-packages/celery/app/trace.py", line 704, in __protected_call__\n return self.run(*args, **kwargs)\n File "/usr/local/lib/python3.6/site-packages/sentry_sdk/integrations/celery.py", line 197, in _inner\n reraise(*exc_info)\n File "/usr/local/lib/python3.6/site-packages/sentry_sdk/_compat.py", line 54, in reraise\n raise value\n File "/usr/local/lib/python3.6/site-packages/sentry_sdk/integrations/celery.py", line 192, in _inner\n return f(*args, **kwargs)\n File "/usr/local/lib/python3.6/site-packages/sentry/tasks/base.py", line 46, in _wrapped\n result = func(*args, **kwargs)\n File "/usr/local/lib/python3.6/site-packages/sentry/tasks/store.py", line 853, in save_event\n _do_save_event(cache_key, data, start_time, event_id, project_id, **kwargs)\n File "/usr/local/lib/python3.6/site-packages/sentry/tasks/store.py", line 767, in _do_save_event\n project_id, assume_normalized=True, start_time=start_time, cache_key=cache_key\n File "/usr/local/lib/python3.6/site-packages/sentry/utils/metrics.py", line 192, in inner\n return f(*args, **kwargs)\n File "/usr/local/lib/python3.6/site-packages/sentry/event_manager.py", line 449, in save\n _get_or_create_release_associated_models(jobs, projects)\n File "/usr/local/lib/python3.6/site-packages/sentry/utils/metrics.py", line 192, in inner\n return f(*args, **kwargs)\n File "/usr/local/lib/python3.6/site-packages/sentry/event_manager.py", line 751, in _get_or_create_release_associated_models\n project=project, release=release, environment=environment, datetime=date\n File "/usr/local/lib/python3.6/site-packages/sentry/models/releaseprojectenvironment.py", line 51, in get_or_create\n release, project, environment, datetime, metrics_tags, **kwargs\n File "/usr/local/lib/python3.6/site-packages/sentry/models/releaseprojectenvironment.py", line 65, in _get_or_create_impl\n defaults={"first_seen": datetime, "last_seen": datetime},\n File "/usr/local/lib/python3.6/site-packages/django/db/models/manager.py", line 82, in manager_method\n return getattr(self.get_queryset(), name)(*args, **kwargs)\n File "/usr/local/lib/python3.6/site-packages/django/db/models/query.py", line 487, in get_or_create\n return self.get(**lookup), False\n File "/usr/local/lib/python3.6/site-packages/django/db/models/query.py", line 397, in get\n num = len(clone)\n File "/usr/local/lib/python3.6/site-packages/django/db/models/query.py", line 254, in __len__\n self._fetch_all()\n File "/usr/local/lib/python3.6/site-packages/django/db/models/query.py", line 1182, in _fetch_all\n self._result_cache = list(self._iterable_class(self))\n File "/usr/local/lib/python3.6/site-packages/django/db/models/query.py", line 53, in __iter__\n results = compiler.execute_sql(chunked_fetch=self.chunked_fetch, chunk_size=self.chunk_size)\n File "/usr/local/lib/python3.6/site-packages/django/db/models/sql/compiler.py", line 1068, in execute_sql\n cursor.execute(sql, params)\n File "/usr/local/lib/python3.6/site-packages/sentry_sdk/integrations/django/__init__.py", line 500, in execute\n return real_execute(self, sql, params)\n File "/usr/local/lib/python3.6/site-packages/django/db/backends/utils.py", line 68, in execute\n return self._execute_with_wrappers(sql, params, many=False, executor=self._execute)\n File "/usr/local/lib/python3.6/site-packages/django/db/backends/utils.py", line 77, in _execute_with_wrappers\n return executor(sql, params, many, context)\n File "/usr/local/lib/python3.6/site-packages/django/db/backends/utils.py", line 85, in _execute\n return self.cursor.execute(sql, params)\n File "/usr/local/lib/python3.6/site-packages/django/db/utils.py", line 89, in __exit__\n raise dj_exc_value.with_traceback(traceback) from exc_value\n File "/usr/local/lib/python3.6/site-packages/django/db/backends/utils.py", line 85, in _execute\n return self.cursor.execute(sql, params)\n File "/usr/local/lib/python3.6/site-packages/sentry/db/postgres/decorators.py", line 77, in inner\n raise_the_exception(self.db, e)\n File "/usr/local/lib/python3.6/site-packages/sentry/db/postgres/decorators.py", line 75, in inner\n return func(self, *args, **kwargs)\n File "/usr/local/lib/python3.6/site-packages/sentry/db/postgres/decorators.py", line 18, in inner\n return func(self, *args, **kwargs)\n File "/usr/local/lib/python3.6/site-packages/sentry/db/postgres/decorators.py", line 95, in inner\n raise exc_info[0](msg).with_traceback(exc_info[2])\n File "/usr/local/lib/python3.6/site-packages/sentry/db/postgres/decorators.py", line 91, in inner\n return func(self, sql, *args, **kwargs)\n File "/usr/local/lib/python3.6/site-packages/sentry/db/postgres/base.py", line 75, in execute\n return self.cursor.execute(sql, clean_bad_params(params))\ndjango.db.utils.ProgrammingError: ProgrammingError(\'column sentry_releaseprojectenvironment.adopted does not exist\\nLINE 1: ...ntry_releaseprojectenvironment"."last_deploy_id", "sentry_re...\\n ^\\n\',)\nSQL: SELECT "sentry_releaseprojectenvironment"."id", "sentry_releaseprojectenvironment"."release_id", "sentry_releaseprojectenvironment"."project_id", "sentry_releaseprojectenvironment"."environment_id", "sentry_releaseprojectenvironment"."new_issues_count", "sentry_releaseprojectenvironment"."first_seen", "sentry_releaseprojectenvironment"."last_seen", "sentry_releaseprojectenvironment"."last_deploy_id", "sentry_releaseprojectenvironment"."adopted", "sentry_releaseprojectenvironment"."unadopted" FROM "sentry_releaseprojectenvironment" WHERE ("sentry_releaseprojectenvironment"."environment_id" = %s AND "sentry_releaseprojectenvironment"."project_id" = %s AND "sentry_releaseprojectenvironment"."release_id" = %s)\n', 'args': '()', 'kwargs': "{'cache_key': 'e:a5a95a97dee44574acac9cb2c1bffed4:3', 'data': None, 'start_time': 1627273310.0, 'event_id': 'a5a95a97dee44574acac9cb2c1bffed4', 'project_id': 3}", 'description': 'raised unexpected', 'internal': False})
- the worker_1 also failed to connect to IP of the public exposed one, which is surely not accessbile ( Client at internet → Firewall–>Our public IP → Firewall–>Intranet for Sentry on-premise, so such public IP is surely not accessible to
worker_1)
orker_1 | urllib3.exceptions.ConnectTimeoutError: (<urllib3.connection.HTTPConnection object at 0x7f16cb172c88>, 'Connection to 140.206.187.251 timed out. (connect timeout=<object object at 0x7f16e96e41f0>)')
worker_1 |
worker_1 | During handling of the above exception, another exception occurred:
worker_1 |
worker_1 | Traceback (most recent call last):
worker_1 | File "/usr/local/lib/python3.6/site-packages/sentry_sdk/transport.py", line 326, in send_event_wrapper
worker_1 | self._send_event(event)
worker_1 | File "/usr/local/lib/python3.6/site-packages/sentry_sdk/transport.py", line 234, in _send_event
worker_1 | headers={"Content-Type": "application/json", "Content-Encoding": "gzip"},
worker_1 | File "/usr/local/lib/python3.6/site-packages/sentry/utils/sdk.py", line 211, in patched_send_request
worker_1 | return _send_request(*args, **kwargs)
worker_1 | File "/usr/local/lib/python3.6/site-packages/sentry_sdk/transport.py", line 180, in _send_request
worker_1 | headers=headers,
worker_1 | File "/usr/local/lib/python3.6/site-packages/urllib3/request.py", line 72, in request
worker_1 | **urlopen_kw)
worker_1 | File "/usr/local/lib/python3.6/site-packages/urllib3/request.py", line 150, in request_encode_body
worker_1 | return self.urlopen(method, url, **extra_kw)
worker_1 | File "/usr/local/lib/python3.6/site-packages/urllib3/poolmanager.py", line 324, in urlopen
worker_1 | response = conn.urlopen(method, u.request_uri, **kw)
worker_1 | File "/usr/local/lib/python3.6/site-packages/urllib3/connectionpool.py", line 667, in urlopen
worker_1 | **response_kw)
worker_1 | File "/usr/local/lib/python3.6/site-packages/urllib3/connectionpool.py", line 667, in urlopen
worker_1 | **response_kw)
worker_1 | File "/usr/local/lib/python3.6/site-packages/urllib3/connectionpool.py", line 667, in urlopen
worker_1 | **response_kw)
worker_1 | File "/usr/local/lib/python3.6/site-packages/urllib3/connectionpool.py", line 638, in urlopen
worker_1 | _stacktrace=sys.exc_info()[2])
worker_1 | File "/usr/local/lib/python3.6/site-packages/urllib3/util/retry.py", line 399, in increment
worker_1 | raise MaxRetryError(_pool, url, error or ResponseError(cause))
worker_1 | urllib3.exceptions.MaxRetryError: HTTPConnectionPool(host='140.206.187.251', port=44552): Max retries exceeded with url: /api/1/store/ (Caused by ConnectTimeoutError(<urllib3.connection.HTTPConnection object at 0x7f16cb172c88>, 'Connection to 140.206.187.251 timed out. (connect timeout=<object object at 0x7f16e96e41f0>)'))
Now no more new events will be logged.
Please kindly check this issue