Upgrade 9.0 with github SSO -> 9.1: Internal server error

Hello everyone,

We have a stable running docker-compose environment running from image sentry:9.0.
It includes the sentry-auth-github provider. Logins are done via github SSO.

After upgrading to sentry:9.1 we encountered an internal server error: ProviderNotRegistered: github

docker-compose.yml:

version: "2"
services:
  redis:
    image: redis:5.0
    restart: always

  postgres:
    user: ${USER_ID}:${GROUP_ID}
    image: postgres:10.10
    volumes:
      - ./data/postgres:/var/lib/postgresql/data
    restart: always

  base:
    build: . # For update change to: image: sentry:9.1
    user: ${USER_ID}:${GROUP_ID}
    environment:
      - SENTRY_SECRET_KEY=${SENTRY_SECRET}
      - SENTRY_REDIS_HOST=redis
      - SENTRY_POSTGRES_HOST=postgres
      - GITHUB_APP_ID=${GITHUB_APP_ID}
      - GITHUB_API_SECRET=${GITHUB_API_SECRET}
    volumes:
      - ./data/sentry:/var/lib/sentry/files

  web:
    extends: base
    links:
      - redis
      - postgres
    environment:
      - VIRTUAL_HOST=sentry.bike.center
      - VIRTUAL_PORT=9000
    ports:
      - 127.0.0.1:9000:9000
    restart: always

  cron:
    extends: base
    command: run cron
    links:
      - redis
      - postgres
    restart: always

  worker:
    extends: base
    command: run worker
    links:
      - redis
      - postgres
    restart: always

Dockerfile:

FROM sentry:9.0
RUN pip install https://github.com/getsentry/sentry-auth-github/archive/master.zip

.env:

SENTRY_SECRET=...

GITHUB_APP_ID=...
GITHUB_API_SECRET=...

USER_ID=1000
GROUP_ID=1000

Question: How to upgrade sentry 9.0 with github SSO provider to sentry 9.1?

Our update processe was:

Logs from docker-compose logs worker:

Attaching to sentry_worker_1
worker_1    | 11:32:21 [WARNING] sentry.utils.geo: settings.GEOIP_PATH_MMDB not configured.
worker_1    | 11:32:33 [INFO] sentry.plugins.github: apps-not-configured
worker_1    | 11:32:33 [INFO] sentry.bgtasks: bgtask.spawn (task_name=u'sentry.bgtasks.clean_dsymcache:clean_dsymcache')
worker_1    |  
worker_1    |  -------------- celery@4b8a2e525e64 v3.1.18 (Cipater)
worker_1    | ---- **** ----- 
worker_1    | --- * ***  * -- Linux-4.15.0-74-generic-x86_64-with-debian-9.11
worker_1    | -- * - **** --- 
worker_1    | - ** ---------- [config]
worker_1    | - ** ---------- .> app:         sentry:0x7f8ef33b7290
worker_1    | - ** ---------- .> transport:   redis://redis:6379/0
worker_1    | - ** ---------- .> results:     disabled
worker_1    | - *** --- * --- .> concurrency: 1 (prefork)
worker_1    | -- ******* ---- 
worker_1    | --- ***** ----- [queues]
worker_1    |  -------------- .> activity.notify  exchange=default(direct) key=activity.notify
worker_1    |                 .> alerts           exchange=default(direct) key=alerts
worker_1    |                 .> app_platform     exchange=default(direct) key=app_platform
worker_1    |                 .> assemble         exchange=default(direct) key=assemble
worker_1    |                 .> auth             exchange=default(direct) key=auth
worker_1    |                 .> buffers.process_pending exchange=default(direct) key=buffers.process_pending
worker_1    |                 .> cleanup          exchange=default(direct) key=cleanup
worker_1    |                 .> commits          exchange=default(direct) key=commits
worker_1    |                 .> counters-0       exchange=counters(direct) key=
worker_1    |                 .> default          exchange=default(direct) key=default
worker_1    |                 .> digests.delivery exchange=default(direct) key=digests.delivery
worker_1    |                 .> digests.scheduling exchange=default(direct) key=digests.scheduling
worker_1    |                 .> email            exchange=default(direct) key=email
worker_1    |                 .> events.index_event_tags exchange=default(direct) key=events.index_event_tags
worker_1    |                 .> events.preprocess_event exchange=default(direct) key=events.preprocess_event
worker_1    |                 .> events.process_event exchange=default(direct) key=events.process_event
worker_1    |                 .> events.reprocess_events exchange=default(direct) key=events.reprocess_events
worker_1    |                 .> events.reprocessing.preprocess_event exchange=default(direct) key=events.reprocessing.preprocess_event
worker_1    |                 .> events.reprocessing.process_event exchange=default(direct) key=events.reprocessing.process_event
worker_1    |                 .> events.save_event exchange=default(direct) key=events.save_event
worker_1    |                 .> files.delete     exchange=default(direct) key=files.delete
worker_1    |                 .> integrations     exchange=default(direct) key=integrations
worker_1    |                 .> merge            exchange=default(direct) key=merge
worker_1    |                 .> options          exchange=default(direct) key=options
worker_1    |                 .> reports.deliver  exchange=default(direct) key=reports.deliver
worker_1    |                 .> reports.prepare  exchange=default(direct) key=reports.prepare
worker_1    |                 .> search           exchange=default(direct) key=search
worker_1    |                 .> sleep            exchange=default(direct) key=sleep
worker_1    |                 .> stats            exchange=default(direct) key=stats
worker_1    |                 .> triggers-0       exchange=triggers(direct) key=
worker_1    |                 .> unmerge          exchange=default(direct) key=unmerge
worker_1    |                 .> update           exchange=default(direct) key=update
worker_1    | 
worker_1    | 11:33:06 [INFO] sentry.plugins.mail: notification.dispatched (project_id=1L plugin=u'mail' event_id=210630L group_id=11162L rule_id=1L digest_key=u'mail:p:1')
worker_1    | celery@4b8a2e525e64 ready.
worker_1    | Traceback (most recent call last):
worker_1    |   File "/usr/local/lib/python2.7/site-packages/celery/app/trace.py", line 240, in trace_task
worker_1    |     R = retval = fun(*args, **kwargs)
worker_1    |   File "/usr/local/lib/python2.7/site-packages/celery/app/trace.py", line 438, in __protected_call__
worker_1    |     return self.run(*args, **kwargs)
worker_1    |   File "/usr/local/lib/python2.7/site-packages/sentry_sdk/integrations/celery.py", line 155, in _inner
worker_1    |     reraise(*exc_info)
worker_1    |   File "/usr/local/lib/python2.7/site-packages/sentry_sdk/integrations/celery.py", line 150, in _inner
worker_1    |     return f(*args, **kwargs)
worker_1    |   File "/usr/local/lib/python2.7/site-packages/sentry/tasks/base.py", line 54, in _wrapped
worker_1    |     result = func(*args, **kwargs)
worker_1    |   File "/usr/local/lib/python2.7/site-packages/sentry/tasks/check_auth.py", line 78, in check_auth_identity
worker_1    |     provider = auth_provider.get_provider()
worker_1    |   File "/usr/local/lib/python2.7/site-packages/sentry/models/authprovider.py", line 47, in get_provider
worker_1    |     return manager.get(self.provider, **self.config)
worker_1    |   File "/usr/local/lib/python2.7/site-packages/sentry/auth/manager.py", line 23, in get
worker_1    |     raise ProviderNotRegistered(key)
worker_1    | ProviderNotRegistered: github
worker_1    | 11:33:34 [ERROR] celery.worker.job: Task sentry.tasks.check_auth_identity[8bafdd99-b45f-4d5f-b4b7-2b9939b86b2f] raised unexpected: ProviderNotRegistered(u'github',) (data={u'hostname': 'celery@4b8a2e525e64', u'name': 'sentry.tasks.check_auth_identity', u'args': '[]', u'internal': False, u'kwargs': "{'auth_identity_id': 6L}", u'id': '8bafdd99-b45f-4d5f-b4b7-2b9939b86b2f'})
[...]

Logs from sentry upgrade:

Creating network "sentry_default" with the default driver
Creating sentry_postgres_1 ... done
Creating sentry_redis_1    ... done
11:31:19 [WARNING] sentry.utils.geo: settings.GEOIP_PATH_MMDB not configured.
11:31:22 [INFO] sentry.plugins.github: apps-not-configured
Syncing...
Creating tables ...
Installing custom SQL ...
Installing indexes ...
Installed 0 object(s) from 0 fixture(s)
Migrating...
Running migrations for sentry:
 - Migrating forwards to 0472_auto__add_field_sentryapp_author.
 > sentry:0424_auto__add_field_integration_status
 > sentry:0425_auto__add_index_pullrequest_organization_id_merge_commit_sha
 > sentry:0425_remove_invalid_github_idps
 > sentry:0426_auto__add_sentryappinstallation__add_sentryapp__add_field_user_is_sent
 > sentry:0427_auto__add_eventattachment__add_unique_eventattachment_project_id_event
 > sentry:0428_auto__add_index_eventattachment_project_id_date_added
 > sentry:0429_auto__add_integrationexternalproject__add_unique_integrationexternalpr
 > sentry:0430_auto__add_field_organizationintegration_status
 > sentry:0431_auto__add_field_externalissue_metadata
 > sentry:0432_auto__add_field_relay_is_internal
 > sentry:0432_auto__add_index_userreport_date_added__add_index_eventattachment_date_
 > sentry:0433_auto__add_field_relay_is_internal__add_field_userip_country_code__add_
 > sentry:0434_auto__add_discoversavedqueryproject__add_unique_discoversavedqueryproj
 > sentry:0435_auto__add_field_discoversavedquery_created_by
 > sentry:0436_rename_projectdsymfile_to_projectdebugfile
 > sentry:0437_auto__add_field_sentryapp_status
 > sentry:0438_auto__add_index_sentryapp_status__chg_field_sentryapp_proxy_user__chg_
 > sentry:0439_auto__chg_field_sentryapp_owner
 > sentry:0440_auto__del_unique_projectdebugfile_project_debug_id__add_index_projectd
 > sentry:0441_auto__add_field_projectdebugfile_data
 > sentry:0442_auto__add_projectcficachefile__add_unique_projectcficachefile_project_
 > sentry:0443_auto__add_field_organizationmember_token_expires_at
 > sentry:0443_auto__del_dsymapp__del_unique_dsymapp_project_platform_app_id__del_ver
 > sentry:0444_auto__add_sentryappavatar__add_field_sentryapp_redirect_url__add_field
 > sentry:0445_auto__add_promptsactivity__add_unique_promptsactivity_user_feature_org
 > sentry:0446_auto__add_index_promptsactivity_project_id
 > sentry:0447_auto__del_field_promptsactivity_organization__add_field_promptsactivit
 > sentry:0448_auto__add_field_sentryapp_is_alertable
 > sentry:0449_auto__chg_field_release_owner
 > sentry:0450_auto__del_grouphashtombstone__del_unique_grouphashtombstone_project_ha
 > sentry:0451_auto__del_field_projectbookmark_project_id__add_field_projectbookmark_
 > sentry:0452_auto__add_field_sentryapp_events
 > sentry:0452_auto__del_field_releaseenvironment_organization_id__del_field_releasee
 > sentry:0453_auto__add_index_releasefile_release_name
 > sentry:0454_resolve_duplicate_0452
 > sentry:0455_auto__add_field_groupenvironment_first_seen
 > sentry:0456_auto__add_dashboard__add_unique_dashboard_organization_title__add_widg
 > sentry:0457_auto__add_field_savedsearch_is_global__chg_field_savedsearch_project__
 > sentry:0457_auto__add_monitorcheckin__add_monitor__add_index_monitor_type_next_che
 > sentry:0458_global_searches_data_migration
Saved Searchs: 100% |#########################################################################################################################################| Time: 0:00:00
 > sentry:0459_global_searches_unique_constraint
 > sentry:0460_auto__add_field_servicehook_organization_id
 > sentry:0461_event_attachment_indexes
 > sentry:0462_auto__add_servicehookproject
 > sentry:0462_releaseenvironment_project_id
 > sentry:0463_backfill_service_hook_project
 > sentry:0464_auto__add_sentryappcomponent__add_field_sentryapp_schema
 > sentry:0464_groupenvironment_foreignkeys
 > sentry:0465_sync
 > sentry:0466_auto__add_platformexternalissue__add_unique_platformexternalissue_grou
 > sentry:0467_backfill_integration_status
 > sentry:0468_auto__add_field_projectdebugfile_code_id__add_index_projectdebugfile_p
 > sentry:0468_recent_search
 > sentry:0469_fix_state
 > sentry:0470_org_saved_search
 > sentry:0471_global_saved_search_types
 > sentry:0472_auto__add_field_sentryapp_author
The following content types are stale and need to be deleted:

    sentry | dsymapp
    sentry | versiondsymfile
    sentry | projectdsymfile
    sentry | grouphashtombstone

Any objects related to these content types by a foreign key will also
be deleted. Are you sure you want to delete these content types?
If you're unsure, answer 'no'.

    Type 'yes' to continue, or 'no' to cancel: yes
Running migrations for sentry.nodestore:
- Nothing to migrate.
Running migrations for sentry.search:
- Nothing to migrate.
Running migrations for social_auth:
- Nothing to migrate.
Running migrations for sentry.tagstore:
- Nothing to migrate.
Running migrations for sentry_plugins.hipchat_ac:
- Nothing to migrate.
Running migrations for sentry_plugins.jira_ac:
- Nothing to migrate.

Synced:
 > django.contrib.admin
 > django.contrib.auth
 > django.contrib.contenttypes
 > django.contrib.messages
 > django.contrib.sessions
 > django.contrib.sites
 > django.contrib.staticfiles
 > crispy_forms
 > debug_toolbar
 > rest_framework
 > sentry.plugins.sentry_interface_types
 > sentry.plugins.sentry_mail
 > sentry.plugins.sentry_urls
 > sentry.plugins.sentry_useragents
 > sentry.plugins.sentry_webhooks
 > sudo
 > south

Migrated:
 - sentry
 - sentry.nodestore
 - sentry.search
 - social_auth
 - sentry.tagstore
 - sentry_plugins.hipchat_ac
 - sentry_plugins.jira_ac
Creating missing DSNs
Correcting Group.num_comments counter

Any help is appreciated.

Hi @Unarz! I think we made a mistake with the documentation as only Google Auth was bundled in at the time of 9.1. GitHub Auth got bundled in only recently so you’d need to upgrade to v10 or install the auth plugin manually like you used to.

/cc @scefali for confirmation. I’ll update the docs page accordingly.

Correct, Github Auth was only added recently

1 Like

Hello @BYK,

we changed the update process to:

  • docker-compose down
  • in Dockerfile change 9.0 to 9.1
  • docker-compose build
  • docker-compose run --rm web upgrade
  • docker-compose up -d

This resolved the issue.

Thank you

1 Like