When will a new version be tagged for Sentry Open Source?

Since this comes up a lot, I want to put all of the information in one place so there’s an easy way to reference people.

tl;dr: sometime in the next few months, when we stabilize our cutover points for new infrastructure

We haven’t tagged a new version of Sentry since 9.0, which is nearly two years ago. Why?

We used to tag versions every month. @matt would sit down and spend a day (or three) going through and making sure the changelog actually represented things, building a bunch of packages, running a bunch of scripts, eventually pushing a new tag to PyPi. This worked somewhat well when the team was 10 folks, as we’d try to get every engineer to write a reasonable changelog entry as part of their PR.

Eventually, @matt and others got busy with, frankly, more important things. Things that let us actually build Sentry and keep it free and open source (e.g. features or other concerns of our paying customers). Additionally we’re far larger than a 10 person team these days. Sentry is now 75 people in two countries and four timezones. The things that allowed us to efficiently keep tagging arbitrary versions of Sentry didn’t scale for long.

We kept saying we’d try to figure out a way to generate a changelog (e.g. automate it), or automate at least tagging of versions, but we never did. The same reason we didn’t do that is the same reason we stopped tagging versions manually: we had other, more important things to do.

Now fast forward to today, and what we’re intending to do:

  1. We’re building a (paid) Single Tenant offering. This will force us to have version points and lag time between upgrades.
  2. We’re hiring a full time Open Source Engineer. They will own the release cycle of Sentry, address concerns brought up by the community, and generally ensure the GitHub issue tracker doesn’t become a shit show.
  3. We’re planning a cutover release (Sentry 9.1, or 10, who knows) which will be the last upgrade before a new versioning scheme.
  4. We’re in the (very early) planning stages of what the new versioning schemes will look like, and how Single Tenant vs Open Source will work.
  5. We almost certainly wont maintain a changelog of features and fixes. We’ll certainly document large infrastructure changes, and upgrade concerns, but we make dozens of changes every week now, and a changelog simply isn’t a good spend of time.

Once we’ve tagged the next version, we’ll be making significant changes to how we release Sentry, as well as a huge set of breaking infrastructure requirements.

I’ve spoken about some of this before, but fundamentally Sentry will stop supporting a lot of custom installation code. Things like MySQL will be completely removed, Snuba (our new Clickhouse-based event storage) will become required, and a lot of the hard-to-maintain freedom will go away. This is also likely when we’ll start making the push towards Django 1.8+ and eventually Python 3.

When will the next version be tagged? We don’t know exactly. We’re trying to finalize some things in Q1, and I’d like to get it done in the same timeframe. That said, we have to get the customer facing changes done before we can take on the secondary (“nice to have”) items. Given that we actually haven’t tagged Sentry in a long time, we’re also mandating a larger QA process to ensure there’s no obvious mishaps (like a broken setup wizard when migrating from 9.0).

I’m happy to answer more questions, but it’s important to remember that Sentry (Open Source) is literally Sentry (the company), and we’re not intentionally crippling the open source product, we just frankly don’t think people should be limiting their ability to use Sentry by requiring arbitrary, blessed-but-not-really, check-in points.

9 Likes

I also want to note that if you’re running 9.0 there SHOULDNT be any required infrastructure changes if you want to switch over to git pinning. Things like ‘tagstore’ (you dont need to know about this) still all run off legacy configuration. This isn’t a fully tested hypothesis of course, so don’t do it on a whim, but the intention is that it’s not a massive upgrade (albeit it has a ton of new code).

Hi Zeeg,

Thank you for explaining the situation. Can you maybe also tell if and how it possible to use git-pinned-based docker instances?

Currently the docs at https://docs.sentry.io/server/installation/docker/ are mentioned as the preferred method of running, but this uses sentry:9.0-onbuild from the Dockerfile.
I’m not very familiar with building Docker images from source, but would that but would that be feasible too?

This should work for building from git: https://github.com/getsentry/docker-sentry/blob/master/build-git.sh

Thanks for the pointer, I get the point.
The script does not run without hickups, but that is another topic, and I deem it to be likely related to me trying to build on Windows.

Can confirm it works.

I went from 9.0 to feat(ui): Add power-features icon (#12040) · getsentry/sentry@e2bfc1f · GitHub and so far it seems to work ( only been running for an hour ).

And you get all the new features so I’m happy :slight_smile:

The script does not run without hickups

True, I had to change the DockerFile because it couldn’t install nodejs due to not finding the proper GPG keys ( timeout issues ).

Just tossed that bit out of there and it built fine afterwards :man_shrugging: . I’m running Docker version 18.06.1-ce, build e68fc7a on Mac

When the script is done you have an image sentry:git-onbuild available which you can use in your own Dockerfile instead of sentry:9.0-onbuild

And then to start run docker run --rm -it <your-own-built-sentry-image> run web --upgrade and you’ll be golden.

Update:
Been running fine for a few days now.

1 Like

Thanks for your update!

With the freshly updated Dockerfile you are referring to, the build succeeds at my end too!
It build took about 10 minutes, and was not without red text (too fast to read), but appeared mostly to be warnings.

Unfortunately when starting the built image I get a one-line error:
standard_init_linux.go:207: exec user process caused "no such file or directory"

This may be due to me building under Windows, while checking the repos with autocrlf, see: https://github.com/docker/labs/issues/215

I will try again!

Just wondering, is the event/discovery part working on master branch? I tried enabling it at sentry.conf but I get the following error when I try to open their page.

HTTPConnectionPool(host='localhost', port=1218): Max retries exceeded with url: /query (Caused by NewConnectionError('&lt;urllib3.connection.HTTPConnection object at 0x7fb534239d50&gt;: Failed to establish a new connection: [Errno 111] Connection refused',))

It is available on master branch, but you’re currently in a lot of undocumented waters here right now. Port 1218 is a service that is required to enable these features called Snuba (https://github.com/getsentry/snuba) and Snuba requires running Zookeeper, Kafka, and ClickHouse along with some other undocumented stuff.

We plan to bundle some stuff here together to make this easier to manage, but we don’t have anything yet at this time.

1 Like

We’ve just tagged v9.1.0 of Sentry, which is our intended final release of the 9.x series, and the last release which will support “mostly whatever infrastructure you want”. We’re hoping this will create a good cutover point for us to migrate off a ton of legacy functionality, which paves the way to some required changes (such as Python 3).

A summary of changes can be viewed here: https://github.com/getsentry/sentry/blob/9.1.0/CHANGES

3 Likes

Super appreciated! What about docker image? I do not see 9.1.0 version on https://hub.docker.com/_/sentry.

Ahh ok, I have found this guy: https://github.com/getsentry/docker-sentry/tree/master/9.1

Gotta wait on this before it’s officially published. :pray:

1 Like

Just wanted to say that I’ve had that official Sentry Docker Hub 9.1 image deployed to my production self-hosted Sentry for about 4 days now. The upgrade from the 9.0 image was pretty painless, other than ensuring I had a lot of our other outdated base infrastructure upgraded (had some PostgreSQL and Redis upgrades knocked out of the way before I attempted the 9.1 upgrade - made sure I was at least running PG 9.6 and Redis 3.2 first).

One problem I have run into is that 9.1 appears to introduce a pretty significant memory leak in the uWSGI web workers that wasn’t present in 9.0. I was forced to configure uWSGI options (in SENTRY_WEB_OPTIONS) to reload workers. There’s a few approaches available to do that, but I found that using reload-on-rss worked the best, only reloading workers when actually needed, and ensuring they didn’t all reload at the same time (as using request count limits, or worker time limits tends to do).

With that configured, it’s still processing roughly 50 million events per month using pretty close to the same amount of system resources.

Thanks for pointing that out!

We’re tracking issues with 9.1 to publish a 9.1.1 release (hopefully nxt week):