Sentry v10 On-Premise Issue Count & Github SSO

Hi,

I’ve been trying to setup Sentry v10 on-premise, I follow the guide in the documentation, and here’s what’s working, and what’s the issue currently I have:

First, Login with Github works, however, there’s a strange behavior that I notice, inside Organization > Auth, there’s a checkbox called “Require SSO”, by default this is enabled, and login screen only present a single button, login with Github
I set this to disable, from what I expect is that now that a form for email login will appear, with login with Github button next to the form, however it’s not the case.
Only login with Github button appear whether I enable or disable this option.
I don’t think it’s the expected behavior, is there any guide to enable both email and SSO login?

Auth config screen:

Login screen:

Now that Sentry is up and running, I try to create project, and send issue, the release is properly created, artifacts are uploaded, issues are sent.
In the release section it properly note that there are two new issues

however, inside detail no issues are recorded

I dig the forum and Github issues, some people suggest to check worker and cron, I did check the queue and the worker

It seems that the worker instance is running 2 worker, and the cron is called from tini

CPU usage hover around ~5% (which is surprisingly low), then I check the queue, in the web it says ~1000jobs started for each hour, and I check task send_email, which says some email are sent?

But even after I set my email notification to “always” I never receive email about issues, however, I do receive important & security email (SSO related, security related, etc) proving that my email credentials are correct.

Any guide or help? where I can check what went wrong? or how do I check the detail for each worker and cron?

Thanks in advance

edit:

  • about email notification, I just read Mail Event-Notifications not working #287, I think that should resolve issue about email, I’ll try it later
  • new user restriction has been lifted, so I edit to make it more clear

Hi @glennKristanto,

Thanks a lot for the very detailed report!

I think this is due to a setting asked you when you first setup Sentry: “Allow registration”. If you disabled that, the form is not displayed. To override this, you can set auth.allow-registration: true in your config.yml file or set SENTRY_OPTIONS['auth.allow-registration'] = True in your sentry.conf.py files and then rebuilding your Sentry containers (docker-compose build web worker cron post-process-forwarder).

I think this and the e-mail issues both stem from the same root cause, which is a service we forgot to include until that GitHub issue you have linked. This caused all post-process task to not being run, causing these kind of issues. Both should be resolved with that fix and getting that is just pulling from our repo and running docker-compose up (you don’t need to run ./install.sh again or rebuild anything). Please let us know if this doesn’t solve the issues.

Thank you for the reply

I see, I did set “Allow Registration” to false.
If we set auth.allow-registration: true then it means anyone can register, what we want to achieve is only allow user creation by invitation only, or using Github SSO, since they’re bound to our organization we assume people outside our organization cannot have access, but using invitation we want to invite other without having them to join our organization, is this will be the case?

I will try it soon.

Oh additional report if I’m allowed.
although I’m not sure if this intended or just bug,
I read that we have to add

# Add plugins here

git+https://github.com/getsentry/sentry-auth-github.git@master#egg=sentry-auth-github

to enable Github SSO, but if we run it just like that, the docker image is missing git to pull it in the repository, so I add command to install git in Dockerfile
also, somehow there’s a script that look for Dockerfile in the base directory instead of sentry/, I haven’t look deep into this (I just add a symlink from sentry/ to base directory)

again thank you for the guide, I will try your suggestion,
then come back with the result

Hi,

We’ve try your suggestion, and here’s the result based on our observation:

Adding the following:

does not make the form appear during login screen, also we see that “Allow Registration” is also available as administrative config, we try to set that to true but also does not make any changes. It’s not that major for us, and we still can use it.

Other issue seems have been resolved in latest revision.

Thanks for the help

Is it possible for you to share your config files, if you have modified them?

Glad to hear!

Sory for the late reply

sure, here’s the content of config.yml
I don’t change much, just fill in credentials

mail.backend: 'smtp'
mail.host: '<host>'
mail.port: 587
mail.username: '<user>'
mail.password: '<password>'
mail.use-tls: true
mail.from: '<sender>'

# If you'd like to configure email replies, enable this.
# mail.enable-replies: true

# When email-replies are enabled, this value is used in the Reply-To header
# mail.reply-hostname: ''

# If you're using mailgun for inbound mail, set your API key and configure a
# route to forward to /api/hooks/mailgun/inbound/
# Also don't forget to set `mail.enable-replies: true` above.
# mail.mailgun-api-key: ''

###################
# System Settings #
###################

# If this file ever becomes compromised, it's important to regenerate your a new key
# Changing this value will result in all current sessions being invalidated.
# A new key can be generated with `$ sentry config generate-secret-key`
system.secret-key: '<secret>'

# The ``redis.clusters`` setting is used, unsurprisingly, to configure Redis
# clusters. These clusters can be then referred to by name when configuring
# backends such as the cache, digests, or TSDB backend.
# redis.clusters:
#   default:
#     hosts:
#       0:
#         host: 127.0.0.1
#         port: 6379

################
# File storage #
################

# Uploaded media uses these `filestore` settings. The available
# backends are either `filesystem` or `s3`.

filestore.backend: 'filesystem'
filestore.options:
  location: '/data/files'
dsym.cache-path: '/data/dsym-cache'
releasefile.cache-path: '/data/releasefile-cache'

system.internal-url-prefix: 'http://localhost:9000'
symbolicator.enabled: true
symbolicator.options:
  url: "http://symbolicator:3021"

transaction-events.force-disable-internal-project: true

github-app.id: <github-app>
github-app.name: '<github-app-name>'
github-app.webhook-secret: '<github-app-webhook-secret>'
github-app.client-id: '<github-app-client-id>'
github-app.client-secret: '<github-app-client-secret>'
github-app.private-key: '<github-app-private-key>'

for sentry.conf.py, I don’t remember changing anything but to add github config,
but here’s the complete content:

from sentry.conf.server import *  # NOQA

DATABASES = {
    "default": {
        "ENGINE": "sentry.db.postgres",
        "NAME": "postgres",
        "USER": "postgres",
        "PASSWORD": "",
        "HOST": "postgres",
        "PORT": "",
    }
}

# You should not change this setting after your database has been created
# unless you have altered all schemas first
SENTRY_USE_BIG_INTS = True

# If you're expecting any kind of real traffic on Sentry, we highly recommend
# configuring the CACHES and Redis settings

###########
# General #
###########

# Instruct Sentry that this install intends to be run by a single organization
# and thus various UI optimizations should be enabled.
SENTRY_SINGLE_ORGANIZATION = True

SENTRY_OPTIONS["system.event-retention-days"] = env('SENTRY_EVENT_RETENTION_DAYS') or 90

#########
# Redis #
#########

# Generic Redis configuration used as defaults for various things including:
# Buffers, Quotas, TSDB

SENTRY_OPTIONS["redis.clusters"] = {
    "default": {
        "hosts": {0: {"host": "redis", "password": "", "port": "6379", "db": "0"}}
    }
}

#########
# Queue #
#########

# See https://docs.getsentry.com/on-premise/server/queue/ for more
# information on configuring your queue broker and workers. Sentry relies
# on a Python framework called Celery to manage queues.

rabbitmq_host = None
if rabbitmq_host:
    BROKER_URL = "amqp://{username}:{password}@{host}/{vhost}".format(
        username="guest", password="guest", host=rabbitmq_host, vhost="/"
    )
else:
    BROKER_URL = "redis://:{password}@{host}:{port}/{db}".format(
        **SENTRY_OPTIONS["redis.clusters"]["default"]["hosts"][0]
    )


#########
# Cache #
#########

# Sentry currently utilizes two separate mechanisms. While CACHES is not a
# requirement, it will optimize several high throughput patterns.

CACHES = {
    "default": {
        "BACKEND": "django.core.cache.backends.memcached.MemcachedCache",
        "LOCATION": ["memcached:11211"],
        "TIMEOUT": 3600,
    }
}

# A primary cache is required for things such as processing events
SENTRY_CACHE = "sentry.cache.redis.RedisCache"

DEFAULT_KAFKA_OPTIONS = {
    "bootstrap.servers": "kafka:9092",
    "message.max.bytes": 50000000,
    "socket.timeout.ms": 1000,
}

SENTRY_EVENTSTREAM = "sentry.eventstream.kafka.KafkaEventStream"
SENTRY_EVENTSTREAM_OPTIONS = {"producer_configuration": DEFAULT_KAFKA_OPTIONS}

KAFKA_CLUSTERS["default"] = DEFAULT_KAFKA_OPTIONS

###############
# Rate Limits #
###############

# Rate limits apply to notification handlers and are enforced per-project
# automatically.

SENTRY_RATELIMITER = "sentry.ratelimits.redis.RedisRateLimiter"

##################
# Update Buffers #
##################

# Buffers (combined with queueing) act as an intermediate layer between the
# database and the storage API. They will greatly improve efficiency on large
# numbers of the same events being sent to the API in a short amount of time.
# (read: if you send any kind of real data to Sentry, you should enable buffers)

SENTRY_BUFFER = "sentry.buffer.redis.RedisBuffer"

##########
# Quotas #
##########

# Quotas allow you to rate limit individual projects or the Sentry install as
# a whole.

SENTRY_QUOTAS = "sentry.quotas.redis.RedisQuota"

########
# TSDB #
########

# The TSDB is used for building charts as well as making things like per-rate
# alerts possible.

SENTRY_TSDB = "sentry.tsdb.redissnuba.RedisSnubaTSDB"

#########
# SNUBA #
#########

SENTRY_SEARCH = "sentry.search.snuba.SnubaSearchBackend"
SENTRY_SEARCH_OPTIONS = {}
SENTRY_TAGSTORE_OPTIONS = {}

###########
# Digests #
###########

# The digest backend powers notification summaries.

SENTRY_DIGESTS = "sentry.digests.backends.redis.RedisBackend"

##############
# Web Server #
##############

SENTRY_WEB_HOST = "0.0.0.0"
SENTRY_WEB_PORT = 9000
SENTRY_WEB_OPTIONS = {
    "http": "%s:%s" % (SENTRY_WEB_HOST, SENTRY_WEB_PORT),
    "protocol": "uwsgi",
    "uwsgi-socket": None,
    "http-keepalive": True,
    "memory-report": False,
}

###########
# SSL/TLS #
###########

# If you're using a reverse SSL proxy, you should enable the X-Forwarded-Proto
# header and enable the settings below

# SECURE_PROXY_SSL_HEADER = ('HTTP_X_FORWARDED_PROTO', 'https')
SESSION_COOKIE_SECURE = True
CSRF_COOKIE_SECURE = True
SOCIAL_AUTH_REDIRECT_IS_HTTPS = True

# End of SSL/TLS settings

############
# Features #
############

SENTRY_FEATURES["projects:sample-events"] = False
SENTRY_FEATURES.update(
    {
        feature: True
        for feature in (
            "organizations:discover",
            "organizations:events",
            "organizations:global-views",
            "organizations:integrations-issue-basic",
            "organizations:integrations-issue-sync",
            "organizations:invite-members",
            "organizations:new-issue-ui",
            "organizations:repos",
            "organizations:require-2fa",
            "organizations:sentry10",
            "organizations:sso-basic",
            "organizations:sso-rippling",
            "organizations:sso-saml2",
            "organizations:suggested-commits",
            "projects:custom-inbound-filters",
            "projects:data-forwarding",
            "projects:discard-groups",
            "projects:plugins",
            "projects:rate-limits",
            "projects:servicehooks",
        )
    }
)

######################
# GitHub Integration #
#####################

GITHUB_APPS_APP_ID = '<github-app-id>'
GITHUB_APP_ID = '<github-app-client-id>'
GITHUB_API_SECRET = '<github-app-client-secret>'
GITHUB_ORGANIZATION = '<github-organization>'
GITHUB_EXTENDED_PERMISSIONS = ['repo']
GITHUB_REQUIRE_VERIFIED_EMAIL = True

This would be problematic if you are using the docker-compose based setup and symbolicator. It should be system.internal-url-prefix: 'http://web:9000'. (this is unrelated to the issues you mentioned tho)

I digged through our code and the behavior related to SSO seems like a bug. I’ll attempt a fix and see how it goes. I’ll link to the PR here once it is up.

I see, thanks for the suggestion

Sure, thanks for the follow up, I’ll check the SSO related directly to Github then

Thanks for the support

1 Like