Relay errors in fresh new on premise install

I did a complete fresh new sentry installation, and got everything works, except… events!

Looking at the logs, when i generate an event i can see it on the web log:

nginx_1 | 138.59.32.18 - - [14/May/2020:13:38:04 +0000] "POST /api/2/store/ HTTP/1.1" 200 41 "-" "sentry.python/0.14.4"

But also, at the same time, the really show some error:

relay_1                    | 2020-05-14T13:38:04Z [relay_server::actors::project_upstream] ERROR: error fetching project states: attempted to send request while not yet authenticated
relay_1                    | 2020-05-14T13:38:06Z [relay_server::actors::project_upstream] ERROR: error fetching project states: attempted to send request while not yet authenticated
relay_1                    | 2020-05-14T13:38:07Z [relay_server::actors::project_upstream] ERROR: error fetching project states: attempted to send request while not yet authenticated
relay_1                    | 2020-05-14T13:38:08Z [relay_server::actors::upstream] ERROR: authentication encountered error: could not send request to upstream
relay_1                    |   caused by: IO error: Connection reset by peer (os error 104)
relay_1                    |   caused by: IO error: Connection reset by peer (os error 104)
relay_1                    |   caused by: IO error: Connection reset by peer (os error 104)
relay_1                    |   caused by: Connection reset by peer (os error 104)
relay_1                    | 2020-05-14T13:38:10Z [relay_server::actors::project_upstream] ERROR: error fetching project states: attempted to send request while not yet authenticated

I guess the problem that event is not showing up is related to this… how can i fix it?

Full relay logs are here:

2020-05-14T13:52:46Z [rdkafka::client] ERROR: librdkafka: Global error: BrokerTransportFailure (Local: Broker transport failure): kafka:9092/bootstrap: Connect to ipv4#172.23.0.10:9092 failed: Connection refused (after 2ms in state CONNECT)
2020-05-14T13:52:46Z [rdkafka::client] ERROR: librdkafka: Global error: AllBrokersDown (Local: All broker connections are down): 1/1 brokers are down
2020-05-14T13:52:46Z [rdkafka::client] ERROR: librdkafka: Global error: BrokerTransportFailure (Local: Broker transport failure): kafka:9092/bootstrap: Connect to ipv4#172.23.0.10:9092 failed: Connection refused (after 10ms in state CONNECT)
2020-05-14T13:52:46Z [rdkafka::client] ERROR: librdkafka: Global error: AllBrokersDown (Local: All broker connections are down): 1/1 brokers are down
2020-05-14T13:52:46Z [relay_server::actors::upstream] ERROR: authentication encountered error: could not send request to upstream
caused by: Failed to connect to host: Failed resolving hostname: no record found for name: web type: AAAA class: IN
caused by: Failed resolving hostname: no record found for name: web type: AAAA class: IN
caused by: Failed resolving hostname: no record found for name: web type: AAAA class: IN
2020-05-14T13:52:46Z [relay_server::actors::upstream] ERROR: authentication encountered error: could not send request to upstream
caused by: Failed to connect to host: Failed resolving hostname: no record found for name: web type: AAAA class: IN
caused by: Failed resolving hostname: no record found for name: web type: AAAA class: IN
caused by: Failed resolving hostname: no record found for name: web type: AAAA class: IN
2020-05-14T13:52:48Z [relay_server::actors::upstream] ERROR: authentication encountered error: could not send request to upstream
caused by: Timeout while waiting for response
2020-05-14T13:52:50Z [relay_server::actors::upstream] ERROR: authentication encountered error: could not send request to upstream
caused by: Failed to connect to host: Connection refused (os error 111)
caused by: Connection refused (os error 111)
caused by: Connection refused (os error 111)
2020-05-14T13:52:52Z [relay_server::actors::upstream] ERROR: authentication encountered error: could not send request to upstream
caused by: Failed to connect to host: Connection refused (os error 111)
caused by: Connection refused (os error 111)
caused by: Connection refused (os error 111)
2020-05-14T13:52:56Z [relay_server::actors::upstream] ERROR: authentication encountered error: could not send request to upstream
caused by: Failed to connect to host: Connection refused (os error 111)
caused by: Connection refused (os error 111)
caused by: Connection refused (os error 111)
2020-05-14T13:53:01Z [relay_server::actors::upstream] ERROR: authentication encountered error: could not send request to upstream
caused by: IO error: Connection reset by peer (os error 104)
caused by: IO error: Connection reset by peer (os error 104)
caused by: IO error: Connection reset by peer (os error 104)
caused by: Connection reset by peer (os error 104)
2020-05-14T13:53:09Z [relay_server::actors::upstream] ERROR: authentication encountered error: could not send request to upstream
caused by: IO error: Connection reset by peer (os error 104)
caused by: IO error: Connection reset by peer (os error 104)
caused by: IO error: Connection reset by peer (os error 104)
caused by: Connection reset by peer (os error 104)
2020-05-14T13:53:20Z [relay_server::actors::upstream] ERROR: authentication encountered error: could not send request to upstream
caused by: IO error: Connection reset by peer (os error 104)
caused by: IO error: Connection reset by peer (os error 104)
caused by: IO error: Connection reset by peer (os error 104)
caused by: Connection reset by peer (os error 104)
2020-05-14T13:53:37Z [relay_server::actors::upstream] ERROR: authentication encountered error: could not send request to upstream
caused by: IO error: Connection reset by peer (os error 104)
caused by: IO error: Connection reset by peer (os error 104)
caused by: IO error: Connection reset by peer (os error 104)
caused by: Connection reset by peer (os error 104)
2020-05-14T13:54:03Z [relay_server::actors::upstream] ERROR: authentication encountered error: could not send request to upstream
caused by: IO error: Connection reset by peer (os error 104)
caused by: IO error: Connection reset by peer (os error 104)
caused by: IO error: Connection reset by peer (os error 104)
caused by: Connection reset by peer (os error 104)
2020-05-14T13:54:13Z [relay_server::actors::project_upstream] ERROR: error fetching project states: attempted to send request while not yet authenticated
2020-05-14T13:54:14Z [relay_server::actors::project_upstream] ERROR: error fetching project states: attempted to send request while not yet authenticated
2020-05-14T13:54:16Z [relay_server::actors::project_upstream] ERROR: error fetching project states: attempted to send request while not yet authenticated
2020-05-14T13:54:18Z [relay_server::actors::project_upstream] ERROR: error fetching project states: attempted to send request while not yet authenticated
2020-05-14T13:54:22Z [relay_server::actors::project_upstream] ERROR: error fetching project states: attempted to send request while not yet authenticated
2020-05-14T13:54:27Z [relay_server::actors::project_upstream] ERROR: error fetching project states: attempted to send request while not yet authenticated
2020-05-14T13:54:35Z [relay_server::actors::project_upstream] ERROR: error fetching project states: attempted to send request while not yet authenticated
2020-05-14T13:54:41Z [relay_server::actors::upstream] ERROR: authentication encountered error: could not send request to upstream
caused by: IO error: Connection reset by peer (os error 104)
caused by: IO error: Connection reset by peer (os error 104)
caused by: IO error: Connection reset by peer (os error 104)
caused by: Connection reset by peer (os error 104)
2020-05-14T13:54:46Z [relay_server::actors::events] ERROR: error processing event: failed to resolve project information
caused by: failed to fetch project state from upstream
2020-05-14T13:54:46Z [relay_server::actors::events] ERROR: error processing event: failed to resolve project information
caused by: failed to fetch project state from upstream
2020-05-14T13:54:58Z [relay_server::actors::project_upstream] ERROR: error fetching project states: attempted to send request while not yet authenticated
2020-05-14T13:54:59Z [relay_server::actors::project_upstream] ERROR: error fetching project states: attempted to send request while not yet authenticated
2020-05-14T13:55:01Z [relay_server::actors::project_upstream] ERROR: error fetching project states: attempted to send request while not yet authenticated
2020-05-14T13:55:03Z [relay_server::actors::project_upstream] ERROR: error fetching project states: attempted to send request while not yet authenticated
2020-05-14T13:55:07Z [relay_server::actors::project_upstream] ERROR: error fetching project states: attempted to send request while not yet authenticated
2020-05-14T13:55:12Z [relay_server::actors::project_upstream] ERROR: error fetching project states: attempted to send request while not yet authenticated
2020-05-14T13:55:20Z [relay_server::actors::project_upstream] ERROR: error fetching project states: attempted to send request while not yet authenticated
2020-05-14T13:55:31Z [relay_server::actors::events] ERROR: error processing event: failed to resolve project information
caused by: failed to fetch project state from upstream
2020-05-14T13:55:31Z [relay_server::actors::events] ERROR: error processing event: failed to resolve project information
caused by: failed to fetch project state from upstream
2020-05-14T13:55:39Z [relay_server::actors::upstream] ERROR: authentication encountered error: could not send request to upstream
caused by: IO error: Connection reset by peer (os error 104)
caused by: IO error: Connection reset by peer (os error 104)
caused by: IO error: Connection reset by peer (os error 104)
caused by: Connection reset by peer (os error 104)
2020-05-14T13:56:39Z [relay_server::actors::upstream] ERROR: authentication encountered error: could not send request to upstream
caused by: IO error: Connection reset by peer (os error 104)
caused by: IO error: Connection reset by peer (os error 104)
caused by: IO error: Connection reset by peer (os error 104)
caused by: Connection reset by peer (os error 104)
2020-05-14T13:57:39Z [relay_server::actors::upstream] ERROR: authentication encountered error: could not send request to upstream
caused by: IO error: Connection reset by peer (os error 104)
caused by: IO error: Connection reset by peer (os error 104)
caused by: IO error: Connection reset by peer (os error 104)
caused by: Connection reset by peer (os error 104)
2020-05-14T13:58:39Z [relay_server::actors::upstream] ERROR: authentication encountered error: could not send request to upstream
caused by: IO error: Connection reset by peer (os error 104)
caused by: IO error: Connection reset by peer (os error 104)
caused by: IO error: Connection reset by peer (os error 104)
caused by: Connection reset by peer (os error 104)

About my setup:
CentOS Linux release 8.1.1911 (Core)
Accessing sentry by external IP and port 9000, without any other service than the one that docker put up
No https

Looks like Relay cannot talk to Kafka or Sentry and this may be due to IPv6. Can you try adding the following under services -> relay in your docker-compose.yml file and see if it helps:

sysctls:
    - net.ipv6.conf.all.disable_ipv6=1

I am experiencing the exact same issue on Ubuntu 18.04.

Adding

sysctls:
    - net.ipv6.conf.all.disable_ipv6=1

under services -> relay in my docker-compose.yml eliminates these DNS-related errors:

caused by: Failed to connect to host: Failed resolving hostname: no record found for name: web type: AAAA class: IN
caused by: Failed resolving hostname: no record found for name: web type: AAAA class: IN
caused by: Failed resolving hostname: no record found for name: web type: AAAA class: IN

However, the main error occurs:

 [relay_server::actors::upstream] ERROR: authentication encountered error: could not send request to upstream
caused by: Failed to connect to host: Connection refused (os error 111)
caused by: Connection refused (os error 111)
caused by: Connection refused (os error 111)

Can you all try this branch and see if it fixes the issues: https://github.com/getsentry/onpremise/pull/499

You’d need to copy my changes in the sentry.conf.example.py into your existing sentry.conf.py file.

Thanks for the response!

Unfortunately, I get the same error:

$ docker-compose logs relay
Attaching to sentry_onpremise_relay_1
relay_1                    | 2020-05-20T13:16:24Z [rdkafka::client] ERROR: librdkafka: Global error: BrokerTransportFailure (Local: Broker transport failure): kafka:9092/bootstrap: Connect to ipv4#172.30.0.10:9092 failed: Connection refused (after 2ms in state CONNECT)
relay_1                    | 2020-05-20T13:16:24Z [rdkafka::client] ERROR: librdkafka: Global error: AllBrokersDown (Local: All broker connections are down): 1/1 brokers are down
relay_1                    | 2020-05-20T13:16:24Z [rdkafka::client] ERROR: librdkafka: Global error: BrokerTransportFailure (Local: Broker transport failure): kafka:9092/bootstrap: Connect to ipv4#172.30.0.10:9092 failed: Connection refused (after 0ms in state CONNECT)
relay_1                    | 2020-05-20T13:16:24Z [rdkafka::client] ERROR: librdkafka: Global error: AllBrokersDown (Local: All broker connections are down): 1/1 brokers are down
relay_1                    | 2020-05-20T13:16:24Z [relay_server::actors::upstream] ERROR: authentication encountered error: could not send request to upstream
relay_1                    |   caused by: Failed to connect to host: Failed resolving hostname: no record found for name: web.REDACTED.com. type: AAAA class: IN
relay_1                    |   caused by: Failed resolving hostname: no record found for name: web.REDACTED.com. type: AAAA class: IN
relay_1                    |   caused by: Failed resolving hostname: no record found for name: web.REDACTED.com. type: AAAA class: IN
relay_1                    | 2020-05-20T13:16:24Z [relay_server::actors::upstream] ERROR: authentication encountered error: could not send request to upstream
relay_1                    |   caused by: Failed to connect to host: Failed resolving hostname: no record found for name: web.REDACTED.com. type: AAAA class: IN
relay_1                    |   caused by: Failed resolving hostname: no record found for name: web.REDACTED.com. type: AAAA class: IN
relay_1                    |   caused by: Failed resolving hostname: no record found for name: web.REDACTED.com. type: AAAA class: IN
relay_1                    | 2020-05-20T13:16:25Z [relay_server::actors::upstream] ERROR: authentication encountered error: could not send request to upstream
relay_1                    |   caused by: Failed to connect to host: Failed resolving hostname: no record found for name: web.REDACTED.com. type: AAAA class: IN
relay_1                    |   caused by: Failed resolving hostname: no record found for name: web.REDACTED.com. type: AAAA class: IN
relay_1                    |   caused by: Failed resolving hostname: no record found for name: web.REDACTED.com. type: AAAA class: IN
relay_1                    | 2020-05-20T13:16:26Z [relay_server::actors::upstream] ERROR: authentication encountered error: could not send request to upstream
relay_1                    |   caused by: Failed to connect to host: Failed resolving hostname: no record found for name: web.REDACTED.com. type: AAAA class: IN
relay_1                    |   caused by: Failed resolving hostname: no record found for name: web.REDACTED.com. type: AAAA class: IN
relay_1                    |   caused by: Failed resolving hostname: no record found for name: web.REDACTED.com. type: AAAA class: IN
relay_1                    | 2020-05-20T13:16:29Z [relay_server::actors::upstream] ERROR: authentication encountered error: could not send request to upstream
relay_1                    |   caused by: Failed to connect to host: Failed resolving hostname: no record found for name: web.REDACTED.com. type: AAAA class: IN
relay_1                    |   caused by: Failed resolving hostname: no record found for name: web.REDACTED.com. type: AAAA class: IN
relay_1                    |   caused by: Failed resolving hostname: no record found for name: web.REDACTED.com. type: AAAA class: IN
relay_1                    | 2020-05-20T13:16:33Z [relay_server::actors::upstream] ERROR: authentication encountered error: could not send request to upstream
relay_1                    |   caused by: Timeout while waiting for response
relay_1                    | 2020-05-20T13:16:38Z [relay_server::actors::upstream] ERROR: authentication encountered error: could not send request to upstream
relay_1                    |   caused by: Failed to connect to host: Connection refused (os error 111)
relay_1                    |   caused by: Connection refused (os error 111)
relay_1                    |   caused by: Connection refused (os error 111)
relay_1                    | 2020-05-20T13:16:46Z [relay_server::actors::upstream] ERROR: authentication encountered error: could not send request to upstream
relay_1                    |   caused by: IO error: Connection reset by peer (os error 104)
relay_1                    |   caused by: IO error: Connection reset by peer (os error 104)
relay_1                    |   caused by: IO error: Connection reset by peer (os error 104)
relay_1                    |   caused by: Connection reset by peer (os error 104)
relay_1                    | 2020-05-20T13:16:57Z [relay_server::actors::upstream] ERROR: authentication encountered error: could not send request to upstream
relay_1                    |   caused by: IO error: Connection reset by peer (os error 104)
relay_1                    |   caused by: IO error: Connection reset by peer (os error 104)
relay_1                    |   caused by: IO error: Connection reset by peer (os error 104)
relay_1                    |   caused by: Connection reset by peer (os error 104)
relay_1                    | 2020-05-20T13:17:14Z [relay_server::actors::upstream] ERROR: authentication encountered error: could not send request to upstream
relay_1                    |   caused by: IO error: Connection reset by peer (os error 104)
relay_1                    |   caused by: IO error: Connection reset by peer (os error 104)
relay_1                    |   caused by: IO error: Connection reset by peer (os error 104)
relay_1                    |   caused by: Connection reset by peer (os error 104)
relay_1                    | 2020-05-20T13:17:23Z [relay_server::actors::project_upstream] ERROR: error fetching project states: attempted to send request while not yet authenticated
relay_1                    | 2020-05-20T13:17:24Z [relay_server::actors::project_upstream] ERROR: error fetching project states: attempted to send request while not yet authenticated
relay_1                    | 2020-05-20T13:17:26Z [relay_server::actors::project_upstream] ERROR: error fetching project states: attempted to send request while not yet authenticated
relay_1                    | 2020-05-20T13:17:28Z [relay_server::actors::project_upstream] ERROR: error fetching project states: attempted to send request while not yet authenticated
relay_1                    | 2020-05-20T13:17:32Z [relay_server::actors::project_upstream] ERROR: error fetching project states: attempted to send request while not yet authenticated
relay_1                    | 2020-05-20T13:17:37Z [relay_server::actors::project_upstream] ERROR: error fetching project states: attempted to send request while not yet authenticated
relay_1                    | 2020-05-20T13:17:40Z [relay_server::actors::upstream] ERROR: authentication encountered error: could not send request to upstream
relay_1                    |   caused by: IO error: Connection reset by peer (os error 104)
relay_1                    |   caused by: IO error: Connection reset by peer (os error 104)
relay_1                    |   caused by: IO error: Connection reset by peer (os error 104)
relay_1                    |   caused by: Connection reset by peer (os error 104)
relay_1                    | 2020-05-20T13:17:45Z [relay_server::actors::project_upstream] ERROR: error fetching project states: attempted to send request while not yet authenticated
relay_1                    | 2020-05-20T13:17:56Z [relay_server::actors::events] ERROR: error processing event: failed to resolve project information
relay_1                    |   caused by: failed to fetch project state from upstream
relay_1                    | 2020-05-20T13:17:56Z [relay_server::actors::events] ERROR: error processing event: failed to resolve project information
relay_1                    |   caused by: failed to fetch project state from upstream
relay_1                    | 2020-05-20T13:17:56Z [relay_server::actors::events] ERROR: error processing event: failed to resolve project information
relay_1                    |   caused by: failed to fetch project state from upstream
relay_1                    | 2020-05-20T13:17:56Z [relay_server::actors::events] ERROR: error processing event: failed to resolve project information
relay_1                    |   caused by: failed to fetch project state from upstream
relay_1                    | 2020-05-20T13:17:56Z [relay_server::actors::events] ERROR: error processing event: failed to resolve project information
relay_1                    |   caused by: failed to fetch project state from upstream
relay_1                    | 2020-05-20T13:17:56Z [relay_server::actors::events] ERROR: error processing event: failed to resolve project information
relay_1                    |   caused by: failed to fetch project state from upstream
relay_1                    | 2020-05-20T13:17:56Z [relay_server::actors::events] ERROR: error processing event: failed to resolve project information
relay_1                    |   caused by: failed to fetch project state from upstream
relay_1                    | 2020-05-20T13:17:56Z [relay_server::actors::events] ERROR: error processing event: failed to resolve project information
relay_1                    |   caused by: failed to fetch project state from upstream
relay_1                    | 2020-05-20T13:17:56Z [relay_server::actors::events] ERROR: error processing event: failed to resolve project information
relay_1                    |   caused by: failed to fetch project state from upstream
relay_1                    | 2020-05-20T13:17:56Z [relay_server::actors::events] ERROR: error processing event: failed to resolve project information
relay_1                    |   caused by: failed to fetch project state from upstream
relay_1                    | 2020-05-20T13:17:56Z [relay_server::actors::events] ERROR: error processing event: failed to resolve project information
relay_1                    |   caused by: failed to fetch project state from upstream
relay_1                    | 2020-05-20T13:17:56Z [relay_server::actors::events] ERROR: error processing event: failed to resolve project information
relay_1                    |   caused by: failed to fetch project state from upstream
relay_1                    | 2020-05-20T13:17:56Z [relay_server::actors::events] ERROR: error processing event: failed to resolve project information
relay_1                    |   caused by: failed to fetch project state from upstream

I tried this with a completely fresh clone (including removing previous volumes and images) on the branch byk/ref/uwsgi, both with and without IPv6 disabled

This indicates that your Docker networking configuration has a problem. web here should resolve to the web service in docker-compose, it should not look up in the public internet but on the internal, sentry_onpremise-specific network.

Do you have any modifications to the network config?

I do not believe I have modifications to the network config.

Here is my docker config and container/network inspect output:

$ git pull
Already up to date.

$ git status
On branch byk/ref/uwsgi
Your branch is up to date with 'origin/byk/ref/uwsgi'.

nothing to commit, working tree clean

$ git diff docker-compose.yml

$ cat /etc/docker/daemon.json
{
  "log-driver": "json-file"
}

$ docker container inspect sentry_onpremise_relay_1
[
    {
        "Id": "de7c5b11ba08e0064b25ba71a05121ab5079be6c973be2380b326993303586ad",
        "Created": "2020-05-20T14:05:52.661832345Z",
        "Path": "/bin/bash",
        "Args": [
            "/docker-entrypoint.sh",
            "run"
        ],
        "State": {
            "Status": "running",
            "Running": true,
            "Paused": false,
            "Restarting": false,
            "OOMKilled": false,
            "Dead": false,
            "Pid": 105550,
            "ExitCode": 0,
            "Error": "",
            "StartedAt": "2020-05-20T14:05:56.056447612Z",
            "FinishedAt": "0001-01-01T00:00:00Z"
        },
        "Image": "sha256:09a4541f6b10120dbfe1623995d02f5af459ba869297e3a148c0a2d7aa9b2435",
        "ResolvConfPath": "/var/lib/docker/containers/de7c5b11ba08e0064b25ba71a05121ab5079be6c973be2380b326993303586ad/resolv.conf",
        "HostnamePath": "/var/lib/docker/containers/de7c5b11ba08e0064b25ba71a05121ab5079be6c973be2380b326993303586ad/hostname",
        "HostsPath": "/var/lib/docker/containers/de7c5b11ba08e0064b25ba71a05121ab5079be6c973be2380b326993303586ad/hosts",
        "LogPath": "/var/lib/docker/containers/de7c5b11ba08e0064b25ba71a05121ab5079be6c973be2380b326993303586ad/de7c5b11ba08e0064b25ba71a05121ab5079be6c973be2380b326993303586ad-json.log",
        "Name": "/sentry_onpremise_relay_1",
        "RestartCount": 0,
        "Driver": "overlay2",
        "Platform": "linux",
        "MountLabel": "",
        "ProcessLabel": "",
        "AppArmorProfile": "docker-default",
        "ExecIDs": null,
        "HostConfig": {
            "Binds": [],
            "ContainerIDFile": "",
            "LogConfig": {
                "Type": "json-file",
                "Config": {}
            },
            "NetworkMode": "sentry_onpremise_default",
            "PortBindings": {},
            "RestartPolicy": {
                "Name": "unless-stopped",
                "MaximumRetryCount": 0
            },
            "AutoRemove": false,
            "VolumeDriver": "",
            "VolumesFrom": [],
            "CapAdd": null,
            "CapDrop": null,
            "Capabilities": null,
            "Dns": null,
            "DnsOptions": null,
            "DnsSearch": null,
            "ExtraHosts": null,
            "GroupAdd": null,
            "IpcMode": "shareable",
            "Cgroup": "",
            "Links": null,
            "OomScoreAdj": 0,
            "PidMode": "",
            "Privileged": false,
            "PublishAllPorts": false,
            "ReadonlyRootfs": false,
            "SecurityOpt": null,
            "UTSMode": "",
            "UsernsMode": "",
            "ShmSize": 67108864,
            "Runtime": "runc",
            "ConsoleSize": [
                0,
                0
            ],
            "Isolation": "",
            "CpuShares": 0,
            "Memory": 0,
            "NanoCpus": 0,
            "CgroupParent": "",
            "BlkioWeight": 0,
            "BlkioWeightDevice": null,
            "BlkioDeviceReadBps": null,
            "BlkioDeviceWriteBps": null,
            "BlkioDeviceReadIOps": null,
            "BlkioDeviceWriteIOps": null,
            "CpuPeriod": 0,
            "CpuQuota": 0,
            "CpuRealtimePeriod": 0,
            "CpuRealtimeRuntime": 0,
            "CpusetCpus": "",
            "CpusetMems": "",
            "Devices": null,
            "DeviceCgroupRules": null,
            "DeviceRequests": null,
            "KernelMemory": 0,
            "KernelMemoryTCP": 0,
            "MemoryReservation": 0,
            "MemorySwap": 0,
            "MemorySwappiness": null,
            "OomKillDisable": false,
            "PidsLimit": null,
            "Ulimits": null,
            "CpuCount": 0,
            "CpuPercent": 0,
            "IOMaximumIOps": 0,
            "IOMaximumBandwidth": 0,
            "Mounts": [
                {
                    "Type": "bind",
                    "Source": "/srv/op9/relay",
                    "Target": "/work/.relay",
                    "ReadOnly": true
                }
            ],
            "MaskedPaths": [
                "/proc/asound",
                "/proc/acpi",
                "/proc/kcore",
                "/proc/keys",
                "/proc/latency_stats",
                "/proc/timer_list",
                "/proc/timer_stats",
                "/proc/sched_debug",
                "/proc/scsi",
                "/sys/firmware"
            ],
            "ReadonlyPaths": [
                "/proc/bus",
                "/proc/fs",
                "/proc/irq",
                "/proc/sys",
                "/proc/sysrq-trigger"
            ]
        },
        "GraphDriver": {
            "Data": {
                "LowerDir": "/var/lib/docker/overlay2/a890af60c3908d876268f296b62f1d7f803eee0f9e3fe6e5031d0ad42e1e9829-init/diff:/var/lib/docker/overlay2/83271fc487bbfcf7413a9cb949153dc6c4eabc865e5f7ff4d9d3d151845831f4/diff:/var/lib/docker/overlay2/c1089fe508709e9134bef7bd24572a8b73c1be9b804a84eb69cbf3da650500d3/diff:/var/lib/docker/overlay2/d11c49e61b272e2b2a8a4b862e12d6e9931302e17a48ca747da2236054d6f07b/diff:/var/lib/docker/overlay2/5725caf7684e2a6f66962881cea9d867555d2bf0ba8584152b3148b524263267/diff:/var/lib/docker/overlay2/697585f69d7b10a745695a92fdfce1d91b0f91756a60507c4162322e6c1d7267/diff:/var/lib/docker/overlay2/189c376721d3d1b443e8a368d9303982e2554f4556f43fe04d2c50c115b24b00/diff:/var/lib/docker/overlay2/3a929ca4029ad189ffa135ad17038605ac6ed2061bc0b0ea06d3051e3a325446/diff",
                "MergedDir": "/var/lib/docker/overlay2/a890af60c3908d876268f296b62f1d7f803eee0f9e3fe6e5031d0ad42e1e9829/merged",
                "UpperDir": "/var/lib/docker/overlay2/a890af60c3908d876268f296b62f1d7f803eee0f9e3fe6e5031d0ad42e1e9829/diff",
                "WorkDir": "/var/lib/docker/overlay2/a890af60c3908d876268f296b62f1d7f803eee0f9e3fe6e5031d0ad42e1e9829/work"
            },
            "Name": "overlay2"
        },
        "Mounts": [
            {
                "Type": "bind",
                "Source": "/srv/op9/relay",
                "Destination": "/work/.relay",
                "Mode": "",
                "RW": false,
                "Propagation": "rprivate"
            },
            {
                "Type": "volume",
                "Name": "c6fdbbea0a7b91dd3171750430b56a42d9b1f14d20f87847d21e898ea35e1e78",
                "Source": "/var/lib/docker/volumes/c6fdbbea0a7b91dd3171750430b56a42d9b1f14d20f87847d21e898ea35e1e78/_data",
                "Destination": "/etc/relay",
                "Driver": "local",
                "Mode": "",
                "RW": true,
                "Propagation": ""
            },
            {
                "Type": "volume",
                "Name": "b43a5d00761e10d41d825e09ff748fb5372c289707de604d558ce5f58bb5987d",
                "Source": "/var/lib/docker/volumes/b43a5d00761e10d41d825e09ff748fb5372c289707de604d558ce5f58bb5987d/_data",
                "Destination": "/work",
                "Driver": "local",
                "Mode": "",
                "RW": true,
                "Propagation": ""
            }
        ],
        "Config": {
            "Hostname": "de7c5b11ba08",
            "Domainname": "",
            "User": "",
            "AttachStdin": false,
            "AttachStdout": false,
            "AttachStderr": false,
            "ExposedPorts": {
                "3000/tcp": {}
            },
            "Tty": false,
            "OpenStdin": false,
            "StdinOnce": false,
            "Env": [
                "PATH=/usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin:/sbin:/bin",
                "RELAY_UID=10001",
                "RELAY_GID=10001"
            ],
            "Cmd": [
                "run"
            ],
            "Image": "getsentry/relay:latest",
            "Volumes": {
                "/etc/relay": {},
                "/work": {}
            },
            "WorkingDir": "/work",
            "Entrypoint": [
                "/bin/bash",
                "/docker-entrypoint.sh"
            ],
            "OnBuild": null,
            "Labels": {
                "com.docker.compose.config-hash": "18d50831cc59fc36392fe1ddda48ef91c21531dcaf5dcf29ce28780bf8a6a251",
                "com.docker.compose.container-number": "1",
                "com.docker.compose.oneoff": "False",
                "com.docker.compose.project": "sentry_onpremise",
                "com.docker.compose.project.config_files": "docker-compose.yml,docker-compose.override.yml",
                "com.docker.compose.project.working_dir": "/srv/op9",
                "com.docker.compose.service": "relay",
                "com.docker.compose.version": "1.25.5"
            }
        },
        "NetworkSettings": {
            "Bridge": "",
            "SandboxID": "0e7900c05d7523533f8ef37de1c85c0a09fd329c47ccb1bed905bfb7ee1fb3e4",
            "HairpinMode": false,
            "LinkLocalIPv6Address": "",
            "LinkLocalIPv6PrefixLen": 0,
            "Ports": {
                "3000/tcp": null
            },
            "SandboxKey": "/var/run/docker/netns/0e7900c05d75",
            "SecondaryIPAddresses": null,
            "SecondaryIPv6Addresses": null,
            "EndpointID": "",
            "Gateway": "",
            "GlobalIPv6Address": "",
            "GlobalIPv6PrefixLen": 0,
            "IPAddress": "",
            "IPPrefixLen": 0,
            "IPv6Gateway": "",
            "MacAddress": "",
            "Networks": {
                "sentry_onpremise_default": {
                    "IPAMConfig": null,
                    "Links": null,
                    "Aliases": [
                        "relay",
                        "de7c5b11ba08"
                    ],
                    "NetworkID": "1ed243b181718d686a1757dc58cd1b850b6147b0fe928b5c8114548fd98e9b56",
                    "EndpointID": "458b767d1c2f76f07db1cd3e2cf8686fad9a45f531ef33ddeddb4ce3f03fb35f",
                    "Gateway": "192.168.48.1",
                    "IPAddress": "192.168.48.11",
                    "IPPrefixLen": 20,
                    "IPv6Gateway": "",
                    "GlobalIPv6Address": "",
                    "GlobalIPv6PrefixLen": 0,
                    "MacAddress": "02:42:c0:a8:30:0b",
                    "DriverOpts": null
                }
            }
        }
    }
]

$ docker network inspect 1ed243b181718d686a1757dc58cd1b850b6147b0fe928b5c8114548fd98e9b56
[
    {
        "Name": "sentry_onpremise_default",
        "Id": "1ed243b181718d686a1757dc58cd1b850b6147b0fe928b5c8114548fd98e9b56",
        "Created": "2020-05-20T10:05:00.625345407-04:00",
        "Scope": "local",
        "Driver": "bridge",
        "EnableIPv6": false,
        "IPAM": {
            "Driver": "default",
            "Options": null,
            "Config": [
                {
                    "Subnet": "192.168.48.0/20",
                    "Gateway": "192.168.48.1"
                }
            ]
        },
        "Internal": false,
        "Attachable": true,
        "Ingress": false,
        "ConfigFrom": {
            "Network": ""
        },
        "ConfigOnly": false,
        "Containers": {
            "14bef6b2e6e3e4d84c1e973d37e3f217ace765246d0206447a0575e3e75d1a68": {
                "Name": "sentry_onpremise_postgres_1",
                "EndpointID": "869aa2e8f498f38f3d1f4ff3f4fa5c4f7a3654e0694bd9293aaa6ed4f33fe8c8",
                "MacAddress": "02:42:c0:a8:30:08",
                "IPv4Address": "192.168.48.8/20",
                "IPv6Address": ""
            },
            "2d16e7f97f12cb8b04e766aa73b4e36d275965cea4c1944929e66fd335778131": {
                "Name": "sentry_onpremise_post-process-forwarder_1",
                "EndpointID": "b935130ed94b81bda019fa40ca29608c7eb2a46c07a8df32ce42239967d032d4",
                "MacAddress": "02:42:c0:a8:30:14",
                "IPv4Address": "192.168.48.20/20",
                "IPv6Address": ""
            },
            "3d9eaec3dd9ae7af231a148304b00c33b2d127517fdca1ec7710c515840d824a": {
                "Name": "sentry_onpremise_snuba-cleanup_1",
                "EndpointID": "323a4017778c41d07142f47d9d54485fa397089f1c2636d914573ffdfe45d7e3",
                "MacAddress": "02:42:c0:a8:30:10",
                "IPv4Address": "192.168.48.16/20",
                "IPv6Address": ""
            },
            "4771ad16d226828ccdf580886178f63a293cc7b517fd0a6bc38f87dbd2161e93": {
                "Name": "sentry_onpremise_kafka_1",
                "EndpointID": "fe009d04b8f0fd6d52b2cf4d37fafd7258e192b6c42c093966130d12bd762dbe",
                "MacAddress": "02:42:c0:a8:30:0a",
                "IPv4Address": "192.168.48.10/20",
                "IPv6Address": ""
            },
            "4ec07d30309a75e879c216c6dcf530facbec615433c9f289e9db00463f79fa09": {
                "Name": "sentry_onpremise_worker_1",
                "EndpointID": "6bd11c8554d79762531ff663c0ff63e7fb5d68b0d558a9f87ac4253fda2fff22",
                "MacAddress": "02:42:c0:a8:30:11",
                "IPv4Address": "192.168.48.17/20",
                "IPv6Address": ""
            },
            "702d611f8ed049d364ed2a0684c960f1b2bfe864eb93c50340da7481286319ed": {
                "Name": "sentry_onpremise_web_1",
                "EndpointID": "4a311cde0d0c4c63874da8bef2755091a7fe5c305ceb6e2c544a14632a8e3857",
                "MacAddress": "02:42:c0:a8:30:12",
                "IPv4Address": "192.168.48.18/20",
                "IPv6Address": ""
            },
            "7622b7dc72d868294ddcea3c3fac1c294e3dbd234615fdf06662ec6a192f8d52": {
                "Name": "sentry_onpremise_symbolicator-cleanup_1",
                "EndpointID": "a9fe56620b75c7cdf66cc4b23fb788373d43738106c1ff5f44f29149b1015c5d",
                "MacAddress": "02:42:c0:a8:30:09",
                "IPv4Address": "192.168.48.9/20",
                "IPv6Address": ""
            },
            "775924bf80c30467e63516df9d083e84178f6f13f812b5dc72537a67bfb45425": {
                "Name": "sentry_onpremise_snuba-api_1",
                "EndpointID": "40534f7f5737a121d5f917905f4090e96d9b4d9878a2315ae730fdcf872cff20",
                "MacAddress": "02:42:c0:a8:30:0c",
                "IPv4Address": "192.168.48.12/20",
                "IPv6Address": ""
            },
            "7cefebb776676447e7a76fc9f5c7364e24ae646b2567449dfa700413cb007f67": {
                "Name": "sentry_onpremise_symbolicator_1",
                "EndpointID": "5931194f794a7b46c3e605d464d2f7600205522bf3131f98d8904ea5c3761a35",
                "MacAddress": "02:42:c0:a8:30:05",
                "IPv4Address": "192.168.48.5/20",
                "IPv6Address": ""
            },
            "8b6a138747b31006cd26082f5d33267e34d6c7217c30afc240e0faddfbc15367": {
                "Name": "sentry_onpremise_smtp_1",
                "EndpointID": "2a8ab5f8d33eb36cdb09bb6686d1061f85c47c0c67fdc51531ae1f7af2982b67",
                "MacAddress": "02:42:c0:a8:30:07",
                "IPv4Address": "192.168.48.7/20",
                "IPv6Address": ""
            },
            "95e3d0183458cd0b3fadba54908673787f29d0763e26d23662559a5b630a9bf1": {
                "Name": "sentry_onpremise_snuba-outcomes-consumer_1",
                "EndpointID": "4bb3114020b6ea7588cd087ed216f92e5cf12a0efa0a2aff48626ded9015bdd8",
                "MacAddress": "02:42:c0:a8:30:0d",
                "IPv4Address": "192.168.48.13/20",
                "IPv6Address": ""
            },
            "960f65dba5585dea07f7d97607acbb861be24eb424f996cf69ca3eafd4a9484d": {
                "Name": "sentry_onpremise_memcached_1",
                "EndpointID": "ee59aca4e947ecd2da9e5cf6a63a9c6b4303f13c85ca4782d152a43e0c5164df",
                "MacAddress": "02:42:c0:a8:30:02",
                "IPv4Address": "192.168.48.2/20",
                "IPv6Address": ""
            },
            "9e654f09cfa28407f82aecc29eaf2894f8dd720805223844db6d57616012363e": {
                "Name": "sentry_onpremise_nginx_1",
                "EndpointID": "f1661772038743e5e2cf3a96c15532e00e0000b9fb18a3a549e8feed8d17f9bf",
                "MacAddress": "02:42:c0:a8:30:17",
                "IPv4Address": "192.168.48.23/20",
                "IPv6Address": ""
            },
            "a309fdd8df23e7c3bb79808aebe40b36ea0655d3755e26588a66c917bb45bec2": {
                "Name": "sentry_onpremise_cron_1",
                "EndpointID": "37c6cde47d832c1d85e6d3da3cd64f2b775ac5eefbd17adef879186ffa8b4fdc",
                "MacAddress": "02:42:c0:a8:30:15",
                "IPv4Address": "192.168.48.21/20",
                "IPv6Address": ""
            },
            "a675e867b7805a6afaea6300284ac8871b873e145a92ca0e9ad9b928cb30aea7": {
                "Name": "sentry_onpremise_ingest-consumer_1",
                "EndpointID": "0a5fdb083cba42ec047a45218d029ebdb53d4d3128b60c3c291b0a2be877a421",
                "MacAddress": "02:42:c0:a8:30:16",
                "IPv4Address": "192.168.48.22/20",
                "IPv6Address": ""
            },
            "b26ce70a974c388a9643ccd301379f9a4acf8ae53b099711c22ec0260d8fabba": {
                "Name": "sentry_onpremise_redis_1",
                "EndpointID": "14dc461bff6eb0089b18c37a70984d2403fe5f58620d13592e29c378ce03735e",
                "MacAddress": "02:42:c0:a8:30:04",
                "IPv4Address": "192.168.48.4/20",
                "IPv6Address": ""
            },
            "c6f2d2f321004b19f1b3aa033116a06c7b361d50cc5abaa516b53495bb41def8": {
                "Name": "sentry_onpremise_zookeeper_1",
                "EndpointID": "6c55f6d3f1448296fe211a309b96244b4456c27f729fec6209ec7f1fd113683e",
                "MacAddress": "02:42:c0:a8:30:03",
                "IPv4Address": "192.168.48.3/20",
                "IPv6Address": ""
            },
            "d1082b16c28e145e1ec2abe6e4429e71c78ab8b45e969502f64b8130f01e11fa": {
                "Name": "sentry_onpremise_sentry-cleanup_1",
                "EndpointID": "779eced8b5a5f6c30ff8be9463c06aa0a82c2ee4d99ecf9cb1fd0eb738ecf7ad",
                "MacAddress": "02:42:c0:a8:30:13",
                "IPv4Address": "192.168.48.19/20",
                "IPv6Address": ""
            },
            "d1569b9e13f6df861c07486e35614e4ba4c2399fc2bcee338a8086359ba21b2b": {
                "Name": "sentry_onpremise_snuba-consumer_1",
                "EndpointID": "770c99cc6b2d7cb3127471ec1e0c79159a9b53eb86ad1adff806c25ece94beca",
                "MacAddress": "02:42:c0:a8:30:0f",
                "IPv4Address": "192.168.48.15/20",
                "IPv6Address": ""
            },
            "de7c5b11ba08e0064b25ba71a05121ab5079be6c973be2380b326993303586ad": {
                "Name": "sentry_onpremise_relay_1",
                "EndpointID": "458b767d1c2f76f07db1cd3e2cf8686fad9a45f531ef33ddeddb4ce3f03fb35f",
                "MacAddress": "02:42:c0:a8:30:0b",
                "IPv4Address": "192.168.48.11/20",
                "IPv6Address": ""
            },
            "e3d7f76ee8adcc5065fcc65ed7a5b22eb383cd78731724e41c9f6bc73bbb6fab": {
                "Name": "sentry_onpremise_snuba-replacer_1",
                "EndpointID": "a8d38f7f4ee09308718400d6cde88f49cd4c5984fef2caf00ba1edb27bb3214b",
                "MacAddress": "02:42:c0:a8:30:0e",
                "IPv4Address": "192.168.48.14/20",
                "IPv6Address": ""
            },
            "f05dce0f4e3cdfe10b83b2b736f4dbe5d04369b7df73a2c9438b712d5fcc5b65": {
                "Name": "sentry_onpremise_clickhouse_1",
                "EndpointID": "13d60a3ef80506ecda66a47d0604d25736f8ccee48c1c3ece2453974a60e3afa",
                "MacAddress": "02:42:c0:a8:30:06",
                "IPv4Address": "192.168.48.6/20",
                "IPv6Address": ""
            }
        },
        "Options": {},
        "Labels": {
            "com.docker.compose.network": "default",
            "com.docker.compose.project": "sentry_onpremise",
            "com.docker.compose.version": "1.25.5"
        }
    }
]

This seems similar to the issue reported here: https://github.com/docker/for-linux/issues/179

Is it possible that you have some DNS resolvers listed in your /etc/resolv.conf that is interfering with Docker’s embedded DNS?