Update 160912: There is a new post here.
We are just about to introduce pull requests at my current position. We are using Git with Atlassian Stash and Jenkins. We want to verify that the pull requests:
- Compile
- Does not break any test cases
- Can be merged to target branch
- Compiles after merge
- Does not break test cases after merge
After some Googling around the issue I found no solution, so I tought I'd make a post about how I solved it.
Verifying source of the pull request
There is a really nice plugin for Jenkins Stash Notifier Plugin that can be used to notify Stash of the status of a build. Enable it on any Jenkins job that builds the branch you are merging from. It will add an icon and a link to Jenkins in the pull request view of Stash.
Discovering new pull requests
I initially solved this with a Jenkins job that is polling Stash for new pull requests. But polling is never good so I created a Stash plugin that will notify Jenkins about new pull requests.
Pull Request Notifier Plugin for Stash
The plugin is available in Atlassian Marketplace and at GitHub. When installed, you will have this configuration GUI.
The features include:
- Trigger on one, or several, event(s) regarding pull requests.
- Invoke one, or several, URL(s) when event(s) are triggered.
- Optionally with basic authentication headers.
- Completely custom URL supporting variable parameters
- ${PULL_REQUEST_ID} Example: 1
- ${PULL_REQUEST_ACTION} Example: OPENED
- ${PULL_REQUEST_AUTHOR_DISPLAY_NAME} Example: Administrator
- ${PULL_REQUEST_AUTHOR_EMAIL} Example: admin@example.com
- ${PULL_REQUEST_AUTHOR_ID} Example: 1
- ${PULL_REQUEST_AUTHOR_NAME} Example: admin
- ${PULL_REQUEST_AUTHOR_SLUG} Example: admin
- ${PULL_REQUEST_FROM_HASH} Example: 6053a1eaa1c009dd11092d09a72f3c41af1b59ad
- ${PULL_REQUEST_FROM_ID} Example: refs/heads/branch_mod_merge
- ${PULL_REQUEST_FROM_REPO_ID} Example: 1
- ${PULL_REQUEST_FROM_REPO_NAME} Example: rep_1
- ${PULL_REQUEST_FROM_REPO_PROJECT_ID} Example: 1
- ${PULL_REQUEST_FROM_REPO_PROJECT_KEY} Example: PROJECT_1
- ${PULL_REQUEST_FROM_REPO_SLUG} Example: rep_1
- And same variables for TO, like: ${PULL_REQUEST_TO_HASH}
You can have several notifications and have them trigger different URL:s. If you trigger Jenkins builds, you may want each repo to have its own build job in Jenkins. The filtering functionality is highly configurable. Create a string with the variables and then a regexp that should match that string.
Polling Jenkins with Groovy script
Note that you should only do it this way if you cannot use the plugin described above! For example, uou may not have enaugh access to Stash to install plugins.
Stash has really nice REST API:s. I created a scheduled job in Jenkins that runs every 5 minutes. I implemented it in Groovy using the Groovy plugin.
- Find all repos: http://stash/rest/api/1.0/projects/PROJECTS/repos/
- Find all pull requests in a repo: http://stash/rest/api/1.0/projects/PROJECTS/repos/"+repo.slug+"/pull-requests?base&details&filterText&orderBy
String summary = ""
int newPullRequests = 0;
File previousPullRequests = new File("/ci/lib/jenkins/workspace/Pull Request Poller/previousPullRequests.txt")
String getJson(String addr) {
manager.listener.logger.println("Getting URL: "+addr)
def authString = "user:pass".getBytes().encodeBase64().toString()
java.net.URLConnection conn = addr.toURL().openConnection()
conn.setRequestProperty( "Authorization", "Basic ${authString}" )
conn.connect()
def reader = new BufferedReader(new InputStreamReader(conn.getInputStream()))
def stringBuilder = new StringBuilder()
String line = null
while ((line = reader.readLine()) != null) {
stringBuilder.append(line + "\n")
}
String json = groovy.json.JsonOutput.prettyPrint(stringBuilder.toString())
manager.listener.logger.println("Got response:\n"+json)
return json
}
new groovy.json.JsonSlurper().parseText(getJson("http://stash/rest/api/1.0/projects/PROJECTS/repos/")).values.each { repo ->
manager.listener.logger.println("Repo: "+repo.slug)
String prettyJSON = getJson("http://stash/rest/api/1.0/projects/PROJECTS/repos/"+repo.slug+"/pull-requests?base&details&filterText&orderBy")
def jsonData = new groovy.json.JsonSlurper().parseText(prettyJSON);
jsonData.values.each { value ->
String title = value.title
String from = value.fromRef.latestChangeset
String fromRepo = value.fromRef.repository.links.clone.find { it.name == "ssh" }.href
String to = value.toRef.latestChangeset
String toRepo = value.toRef.repository.links.clone.find { it.name == "ssh" }.href
String repositorySlug = repo.slug
String pullRequestId = value.id
String requestUrl = "http://stash/projects/PROJECTS/repos/"+repositorySlug+"/pull-requests/"+pullRequestId+"/overview"
//Remember that this request has been triggered, and avoid triggering it again
String identifier = from+" "+to
if (previousPullRequests.text.contains(identifier)) {
manager.listener.logger.println("Ignoring: "+identifier)
return;
}
previousPullRequests.append(identifier+"\n")
//Trigger a jenkins job that will verify the pull request
String invokeBuildUrl = "http://jenkins/job/Pull%20Request%20Builder/buildWithParameters?token=SECRET_CONFIGURED_IN_BUILD&FROM="+from+"&TO="+to+"&FROMREPO="+fromRepo+"&TOREPO="+toRepo+"&REPOSITORY_SLUG="+repositorySlug+"&PULL_REQUEST_ID="+pullRequestId
manager.listener.logger.println(invokeBuildUrl)
new URL(invokeBuildUrl).getText()
summary += "<h1>"+title+"</h1><br><a href='"+requestUrl+"'>"+requestUrl+"</a><br>From: "+jsonData.values[0].fromRef.id+" ("+from+") in "+fromRepo+"<br>To: "+jsonData.values[0].toRef.id+" ("+to+") in "+toRepo+"<br><a href='"+invokeBuildUrl+"'>"+invokeBuildUrl+"</a><hr>"
newPullRequests++;
}
}
//Add some info to the build
if (newPullRequests == 0) {
manager.createSummary("gear2.gif").appendText("<h1>No new pull requests found!</h1>" , false)
} else {
manager.addShortText("+"+newPullRequests, "grey", "white", "0px", "white")
manager.createSummary("gear2.gif").appendText(summary , false)
}
Merging and building the pull request
I created a parameterized job to merge the pull request from source branch to target branch. It takes FROM_HASH, FROM_REPO, TO_HASH, TO_REPO, REPOSITORY_SLUG and PULL_REQUEST_ID as parameters.
The job has a build step execute shell that does the actual verification.
git clone $TO_REPO
cd *
git reset --hard $TO_HASH
git status
git remote add from $FROM_REPO
git fetch from
git merge $FROM_HASH
git --no-pager log --max-count=10 --graph --abbrev-commit
#compile command here ...
The job uses the Stash Notifier Plugin to record result in the pull request in Stash. Use the ${FROM_HASH} variable to get the build status reported correctly in the pull request in Stash.
It adds a comment to the pull request, like this.
curl -D- -u user:pass -X POST -H "Content-Type: application/json" --data "{ \"text\": \"Looking good :) http://jenkins/job/Pull%20Request%20Builder/${BUILD_NUMBER}/\" }" http://stash/rest/api/1.0/projects/PROJECT/repos/$REPOSITORY_SLUG/pull-requests/$PULL_REQUEST_ID/comments
Static code analyzers
If you are using static code analyzers you may want to have a look at Jenkins Violation Comments to Stash Plugin for Jenkins.
It is configured like this.
And will comment the pull requests like this.