Hello,
i tried to install sentry on openshift. The interface works great, the events that i emmit are received but it seems that the events are not processed by the workers. I have a message on the interface: Background workers haven’t checked in recently. It seems that you have a backlog of 146 tasks. Either your workers aren’t running or you need more capacity. And on the administration/ menu no jobs was started.
Can you help me on what to check in order to solve my problem please?
Regards
As it says, it sounds like workers aren’t running. You need to make sure you have a sentry run worker and sentry run cron process.
Hello,
i did it as it is writen on the documentation.
regards
Then I’m not sure. Sounds like you have some misconfiguration. I’d check by increasing the log level on the worker and see what it’s doing.
is there any logs or command i can play in order to see what is wrong?
sentry run worker --loglevel=info or sentry run worker --loglevel=debug, and logs are spit out to stderr/stdout. So you can just run this in foreground to get an idea of what’s going on.
Ok, thank you. I runned cron and worker command with the option loglevel and i don’t see something wrong in logs.
In fact, events are well received because the POST request from the client is 200 OK. In the sentry web app i can see in stats accepted events and api responses. I also can see on the main page on top right the events graph is ok.
But i can’t see this events on the main page “New this week” or in the project page so i have no informations on these events.
Can you tell me what i could check in order to solve my problem please?
ok i found my error, the command sentry run worker didn’t work because i didn’t set the C_FORCE_ROOT env variable. Now it is ok. Thank you for your help
regards
You shouldn’t be running this as root, but ultimately that’s your decision.
I am experiencing the same issue.
Am I running things in the incorrect order? For example, I SSH into the server and run “sentry run web” then “sentry run worker” then “sentry run cron”
After running the web command I only ever see log output for the web, so can’t tell if the worker is running.
Can you provide a little more assistance? Is this how the commands should be run?
I also, with my very limited knowledge with Linux, made a shell script to run those commands. Same thing, web runs but i get the message about the workers having not checked in.
Do you have everything running and configured from here: https://docs.sentry.io/server/installation/ ?
Yes. All up and running. Just the workers not appearing to be running.
Now - and this is where I think it’s my lack of knowledge in Linux. Anything I run after “sentry run web” isn’t actually being run because the web process seems to have focus on my terminal.
If i, for example, run the worker first and then web, i get no web. If i run web first then worker i get no worker.
Am i supposed to run these commands in separate terminal windows? I really have no idea what i am doing when it comes to Linux 
Interestingly, when i run those commands in their own terminals i see the workers spin up (just from log output) yet the message on the web interface persists.
If your Linux experience is low. I would highly suggest the Docker install: https://docs.sentry.io/server/installation/docker/
Everything is already there and ready to go.
Honestly, that might be the best route forward but doesn’t address the issue at hand. It also won’t help me understand Linux a little more.
Everything is running (bar the workers), and it’s surely some sort of config issue. Myself and the OP can’t be the only people to ever have this issue.
@Darrenthebrit your issue is entirely different from OP most likely. If you’re new to Linux, getting up and running with Sentry is frankly one of the more difficult things you can start with. And based on yiur questions, you need to learn some fundamentals about the command line first.
I don’t really enjoy not being constructive, but I can’t start from the beginning and teach how computing works and how to run programs from the command line. Your issue is just a fundamental misunderstanding that will become obvious later.
@Darrenthebrit if you cannot use the recommended Dockerized install procedure, the following might help:
Even if you don’t use the Debian package, the contained systemd units & launcher scripts help to run the services in a robust way (running them in SSH prompts is not).
@matt - if it helps, I have been a software developer for over 17 years. Just not Linux. So the windows/linux crossover gets things a little muddled. And what is it that has you thinking it’s most likely a different issue? I’m also not looking for lessons in how computers work.
@jhermann - thanks. Again though, removing something and putting something else in its place because of, what likely comes down to a configuration issue is frankly odd. It seems that every time I venture into Linux-land and come across a problem the answer is pretty much always “delete it and start again with this, or that” - as opposed to helping figure out the actual underlying problem; this is something that keeps me away from open source; rightly or wrongly.
So what this boils down to is: Are there some things I can do to validate my install. Changing config for more logs etc.
I cannot believe the only way to resolve an install issue is to delete something, start again and just pray to god that i don’t have an issue on the other 2 options. One of my most upvoted questions (and also my own answer) on Stack Overflow was because I missed a step in something. It turns out that thousands of other .Net developers had done the same thing. Not one person recommended dumping the idea in place of something else because a step was missed.
Well, harsh truth, it boils down to you being able to debug what you have (you said you’re not, and that’s ok), or use proven solutions by people that know Linux.
Remotely debugging a broken setup, with very little to no info on what happened to arrive at it, is not time-effective for anyone.
PS: Starting over in a VM or Docker instance is cheap these days, no need to throw away what you have just to try something else.
I’m not asking for remote debugging. Asking for guidance.
We already know that I have ran the commands, as mentioned. I have tried them in various orders too. It all comes down to one issue - the workers are apparently not running. So what are the possible causes of this? So far, all i have seen is 'you need to “sentry run worker” ’ - well, i have. So what else could result in the workers not appearing to be running - seems like it could be a config issue to me. If so, where do i look and what do i look for.
And yes, starting again is cheap-ish. Having an issue that can’t be resolved through one reason or another is freakin expensive.
You need to run all the processes concurrently. Not just one at a time. As you’re mentioning, you’re trying to just run each process in the foreground without any process manager to background and manage them as a daemon. This is documented here: https://docs.sentry.io/server/installation/python/#running-sentry-as-a-service if you’re not using docker. If you use docker, that is kinda built in.