Sentry.io ~ All commits associated to all releases

We have a server running a laravel app. It receives deploys through envoyer.io, and using the deployment hooks, on each deploy we generate a new release and a new deploy attached to it.

We do it by using curl as follows:

export SENTRY_BEARER_TOKEN="{{sentry's bearer token here}}"
export SENTRY_PROJECT="{{projectName}}"
export ENVIRONMENT="develop"
export PREVIOUS_SHA=`tail {{release}}/.commit_hash_previous`

cd {{ release }}
curl https://sentry.io/api/0/organizations/{{organization}}/releases/ \
-H "Authorization: Bearer ${SENTRY_BEARER_TOKEN}" \
-X POST \
-H "Content-Type:application/json" \
-d "{
    \"environment\":\"${ENVIRONMENT}\",
    \"version\":\"{{sha}}\",
    \"refs\":[{
        \"repository\":\"{{repository_name}}\",
        \"commit\":\"{{sha}}\",
        \"previousCommit\": \"${PREVIOUS_SHA}\"
    }],
    \"projects\": [\"{{sentry_project_slug}}\"]
}"

curl https://sentry.io/api/0/organizations/{{organization}}/releases/{{sha}}/deploys/ \
-X POST \
-H "Authorization: Bearer ${SENTRY_BEARER_TOKEN}" \
-H 'Content-Type: application/json' \
-d "
{
    \"environment\": \"${ENVIRONMENT}\",
    \"name\": \"{{release}}\"
}"

The above code runs on each deployment to each environment (develop and production).

And so far, it seems to be working… I am getting the Releases tab populated, with releases that correspond to the sha of the commit that triggers each deployment… This is, pretty much, what we intended… However the distribution of commits to these releases/deploys does not seem to be correct.

We end up with something like this:


Looking like there are 90 or so commits on each deploy, which is strange, as a lot of these deploys (in the screenshot) were hotfixes on a file while I was tweaking the envoyer hooks for these things to get logged properly…

I managed to make it show the environment of the deployment, and the sha of the commit in the release, but the commits… I need help with that part… for I am totally lost.

Edit:
Forgot to add, our repository is on bitbucket. It is a private repo. We had it added in the organization settings. Is there anywhere else where one must add it!? If yes, then maybe I was missing that!? Although, the releases feature kinda behaves as though it has access to the repository … too much access even…

Is it possible PREVIOUS_SHA is incorrect? You can additionally try removing previousCommit from the payload and Sentry will automatically pick up the sha from the previous release.

The way we pull the commits is by hitting:

https://api.bitbucket.org/2.0/repositories/:repo_name/commits/:commit_head

We then paginate through until we find previousCommit (or until we hit 90 results).

That’s the thing.

What we do in “PREVIOUS_SHA” is on an earlier hook in envoyer, we write to file the sha of the commit that triggered the release. Then before envoyer changes the symlink and flips the current folder to the new release, we read the current sha we previously saved to file on the previous release, and we save it to previous_sha file. We then read it using tail and send it via curl. For clarity, these are the lines that do it:

yes | cp -f {{project}}/current/.commit_hash {{release}}/.commit_hash_previous
echo "{{ sha }}" > {{release}}/.commit_hash

This way, we have two dot files at the root of each release which contains the sha of the commits that 1. trigger the current release, and 2. the one that triggered the previous release.

I redacted it from the opening post, but I am also echoing out these values, and they seem correct when compared to my git history on bitbucket.

Without the previousCommit in the json payload, I was still getting the same 90 commits there… This is why I feel there is something wrong… somewhere… but I’m not too sure where.

I can fetch the values of the responses I get when creating the release and the subsequent deploy curl call if it could help checking things out…

  % Total    % Received % Xferd  Average Speed   Time    Time     Time  Current
                             Dload  Upload   Total   Spent    Left  Speed

  0     0    0     0    0     0      0      0 --:--:-- --:--:-- --:--:--     0
100  2356{"dateReleased": null, "commitCount": 90, "url": null, "data": {}, "lastDeploy": {"name": "{{envoyer_path_to_release_folder}}", "url": null, "environment": "develop", "dateStarted": null, "dateFinished": "2018-05-29T14:25:12.863Z", "id": "661015"}, "deployCount": null, "dateCreated": "2018-05-28T23:38:53.825Z", "lastEvent": null, "version": "555005178f38", "firstEvent": null, "lastCommit": {"repository": {"status": "active", "name": "{{repository_name}}", "url": "{{repository_url}}", "dateCreated": "2017-07-24T19:21:08.246Z", "provider": {"id": "bitbucket", "name": "Bitbucket"}, "id": "2268"}, "releases": [{"dateReleased": null, "url": null, "dateCreated": "2018-05-28T23:38:53.825Z", "version": "555005178f38", "shortVersion": "555005178f38", "ref": null}, {"dateReleased": null, "url": null, "dateCreated": "2018-05-29T14:23:19.511Z", "version": "05c95054e510", "shortVersion": "05c95054e510", "ref": null}], "author": {"name": "{{another_redacted_name}}", "email": "{{another_redacted_email}}"}, "dateCreated": "2018-05-28T23:25:33Z", "message": "Merge branch 'hotfix/0.108.17' into develop", "id": "555005178f385c65f35450a96db3dac268c8ff86"}, "shortVersion": "555005178f38", "authors": [{"username": "{{verified_user_1_username}}", "lastLogin": "2018-05-01T12:53:48.954Z", "isManaged": false, "lastActive": "2018-05-29T14:30:23.421Z", "id": "120302", "isActive": true, "has2fa": false, "name": "{{verified_user_1}}", "avatarUrl": "{{gravatar_url}}", "dateJoined": "2017-03-06T14:48:15.551Z", "emails": [{"is_verified": true, "id": "118207", "email": "{{verified_email_1}}"}], "avatar": {"avatarUuid": null, "avatarType": "letter_avatar"}, "hasPasswordAuth": true, "email": "{{another_email_here}}"}, {"name": "{{author_3_name}}", "email": "{{author_3_email}}"}, {"name": "{{author_1_name}}", "email": "{{author_1_email}}"}, {"name": "{{author_2_name}}", "email": "{{author_2_email}}"}, {"name": "{{author_4_name}}", "email": "{{author_4_email}}"}], "owner": null, "newGroups": 0, "ref": null, "projects": [{"slug": "{{project_slug}}", "name": "{{project_name}}"}]}  100  2125  100   231   5919    643 --:--:-- --:--:-- --:--:--  5935

The Deploy

  % Total    % Received % Xferd  Average Speed   Time    Time     Time  Current
                                 Dload  Upload   Total   Spent    Left  Speed

  0     0    0     0    0     0      0      0 --:--:-- --:--:-- --:--:--     0
100   293  100   188  10{"name": "{{envoyer_path_to_release_folder}}", "url": null, "environment": "develop", "dateStarted": null, "dateFinished": "2018-05-29T14:33:52.909Z", "id": "661029"}0   105    582    325 --:--:-- --:--:-- --:--:--   582
100   293  100   188  100   105    579    323 --:--:-- --:--:-- --:--:--   578

Some info redacted from responses for privacy.

Could it be that the hash I’m using is not the full-hash (it is shorter/sub-string than the full-length sha from bitbucket) and when compared to what’s in the returned data from that api endpoint, (i believe I saw full-length hashes) my provided string doesn’t match!?

When I ask envoyer for the sha it gives me a substring of the full sha passed by bitbucket. An example from my current repository:

61592e67b362

That endpoint you refer to provides a lot of matches (using chrome’s cmd+f search function) to this substring, but only because it searches fuzzy-like… The full sha is:
61592e67b3621b009dabf883aff5169d4b070c01

If the matching on your end is being done 1:1 then I will not provide something that gives us a proper match. If my substring of the sha will be matched to the full-length sha … this could be the case here, yes?

Usually the APIs accept a partial hash, and then we look that up with the remote service (e.g. BitBucket). Assuming that endpoint I listed above accepts a reference/partial SHA I dont know why it wouldn’t work. It’s a bit diff than GitHub though, as GitHub I think we can send any reference and are able to look up the details.

It’s like a huge gray area to me… if it does… accept a partial sha or whatever it is accepting… they call it “revision” and the specific parameter has no description in their api documentation…

https://developer.atlassian.com/bitbucket/api/2/reference/resource/repositories/{username}/{repo_slug}/commits/{revision}

Ugh… I’m sure things like these are why bitbucket is a pain to work with…

I’m outta things to try now…

I opened a support ticket with the zendesk thingy… I uploaded actual hashes and repository names in case it helps debug on your end. Also a couple screenshots.

Hi @Effectix - I think that due to a bug in our plugin, we relied on full hashes in this loop:

I opened a PR to resolve this here, it should be shipped by tomorrow: https://github.com/getsentry/sentry-plugins/pull/362

1 Like

I am very glad to inform that after seeing the merge to master of the afore mentioned fix, we are now seeing the expected number of commits on our releases.

Thanks a lot guys! Keep up the great stuff!

cheers! thank you for following up!