Integrate CodeScene with Pull Requests

CodeScene offers an automated integration with GitHub, BitBucket, Azure DevOps or GitLab pull requests to incorporate the analysis results into existing delivery workflows.

Integration is different per provider:

  • Github via Checks

  • Bitbucket via either Builds or Code Insights Reports

  • Azure Status Check and Pull Request comment

  • GitLab via Merge Request Discussion comment

See Bitbucket, Azure, GitLab presentation.

CodeScene and Pull Requests: Use Cases

The purposes of CodeScene’s pull request integration is to:

  • Specify quality gates that trigger in case the Code Health of a hotspot declines.

  • Serve as a quality gate and provide feedback on user-defined hotspot goals (e.g. Supervise, Planned Refactoring).

  • Prioritize code reviews based on the risk of the commits.

  • Get early warnings on new and complex code, and detect the absence of expected change coupling to catch omissions.

CodeScene's quality gate triggers in a GitHub check on code health decline.

Fig. 85 CodeScene’s quality gate triggers in a GitHub check on code health decline.

Example on the review recommendations: CodeScene notifies reviewers when critical code with potential security implications is modified.

Fig. 86 Example on the review recommendations: CodeScene notifies reviewers when critical code with potential security implications is modified.

Quality Gates for declining Code Health

CodeScene integrates with pull requests via GitHub’s checks API. That way, CodeScene can act as a (soft) quality gate during development. If a hotspot declines in health, CodeScene detects it, fails the checks, and presents the details:

Example on a failing quality gate where a hotspot declines in code health.

Fig. 87 Example on a failing quality gate where a hotspot declines in code health.

Prioritize Code Reviews via the predicted Delivery Risk

In many organizations, code reviews tend to become bottlenecks. There’s only so much code we can review each day, and after a while it becomes easy to slip. Suddenly, a critical error makes its way to production.

To help optimize code reviews, CodeScene recommends a code review level based on the predicted delivery risk of the PR:

Prioritize code reviews based on the predicted delivery risk.

Fig. 88 Prioritize code reviews based on the predicted delivery risk.

Use the recommended review level as a guideline to save time during code reviews.

How are high risk PRs identified?

So how can CodeScene predict the necessary review level? Under the hood, CodeScene calculates a unique risk profile for your codebase based on how the system has evolved. This risk profile is a combination of technical and social metrics. The technical metrics relate to the depth and diffusion of the change – the more changes and the more widespread, the higher the risk.

The social dimension of the risk profile relates to the experience of the programmer doing the change. Note that experience is relative to a specific repository, and measured as how much each programmer has contributed to your code historically.

Finally, the emphasize is on _prediction_; while it’s likely that a detected high risk change contains a bug, the tool cannot guarantee it, not can it guarantee that lower risk changes are bug free. It’s all about probabilities.

Enable the Pull Request Integration

Navigate to your project’s configuration in CodeScene, navigate to Pull Request Integration settings.

If you’re using GitHub the first step is to install the GitHub app for the integration. CodeScene will inform you of this step, if needed:

Install the GitHub integration for pull request integrations.

Fig. 89 Install the GitHub integration for pull request integrations.

With the app installed, all you need to do is to check a box and you are up and running with a single click:

Enable the pull request integration with a single click.

Fig. 90 Enable the pull request integration with a single click.

If you are using Bitbucket you’ll need to install an Atlassian Connect App on the Workspaces of repositories of the CodeScene project, there is a link on the configuration page to install it.

Pull Request Analysis Results Presentation

To see how results are presented as GitHub checks see the images above. This section shows presentation in other providers:

Bitbucket result presentation

The default presentation with Bitbucket is the Build option. Analysis results will be presented in a Pull Request comment and as a Build status. The Build status can be made a requirement for merging the Pull Request.

Alternatively you can get a Code Insights Report, which will not prevent merges if you have configured successful builds to be a requirement.

Bitbucket PR comment

Fig. 91 Example of a Pull Request comment in Bitbucket

Bitbucket PR report

Fig. 92 Code Insights Report is located bottom of the right sidebar of Bitbucket PR

Bitbucket PR report

Fig. 93 Build Check in the right is located in the middle of the right sidebar of Bitbucket PR

Azure result presentation

Results are presented as status check and a PR comment.

Azure Status Check

Fig. 94 A failed Status Check

You can make the check mandatory for merging into a specific branch:

  • go to Azure’s Project Settings

  • Repositories

  • select a repository

  • Policies tab

  • select “master” or similar main branch in Branch Policies

  • Status Checks, click + button to add one

On this same page you can also enable: Check for comment resolution. Since CodeScene’s PR comments on failed checks are Unresolved, this setting forms a Quality Gate that prevents PR merging, unless PR comment is manually marked as resolved. This gives a nice flexibility where CodeScene’s failed PR check can be manually overridden.

GitLab result presentation

Results are presented as Discussion comment in the Merge Request:

GitLab MR Comment

Fig. 95 A successful check

If PR check is successful then the comment is resolved, otherwise it’s unresolved. Status is shown with a green checkmark (red arrow points to it).

This is useful if you set the restriction to prevent merge if there are unresolved comments.

Set Up an Automatic Analysis Schedule

CodeScene will always try to fetch the latest version of every file changed in a pull request, but it won’t do that for large pull requests containing more than 30 files.

To get the best results for the Pull Request Integration checks, it’s a good idea to set up an automatic full analysis for your project.

You can do this in the Analysis schedule section of the project configuration. See the Getting Started section to learn more about this feature.

Pull Requests: Statistics, Actions, and Impact

No matter what baseline we start from, we never want our code to become harder to read, understand, or maintain. CodeScene’s pull request integration lets you stay on top of your development so that you can prioritize and act upon any negative trends. For that purpose, CodeScene visualizes the impact of pull requests over time:

View the impact and developer action due to the pull request integration.

Fig. 96 View the impact and developer action due to the pull request integration.

The presented statistics are:

  • Checks performed: This is the total number of pull request analyses run by CodeScene.

  • Detected Violations: A violation is either a violated goal (e.g. a planned refactoring never happened), a reported code health decline, or new code with code health issues.

  • Positive Actions: A positive action is when one of the detected violations is acted upon and mitigated.

  • Violations Ignored: This is the number of detected violations that were merged as-is, without remediation, to the main branch. Ignored violations lead to a failed goal and/or a code health decline.

CodeScene also calculates the total code health impact and presents it as monthly trends. These trends let you visualize and communicate improvements, as well as taking actions upon a larger decline:

View the code health impact of the pull request integration.

Fig. 97 View the code health impact of the pull request integration.

In the preceding example, we see that Codebase A took on a significant amount of technical debt during November. This code health decline would also be reported as alerts on the CodeScene dashboard when the pull requests are merged. A recommended action in this case would be to plan goals to manage the new debt (see Manage Hotspots and Technical Debt with Goals).

Contrast this with Codebase B which shows a positive trend where smaller code improvements are delivered continuously.

Finally, you can inspect the details of individual pull requests if you need to drill down even further:

CodeScene presents detailed statistics per pull request, allowing a deeper drill down.

Fig. 98 CodeScene presents detailed statistics per pull request, allowing a deeper drill down.

You can find the pull request statistics on your project’s analysis dashboard:

The dashboard includes pull request statistics and a link to view more details.

Fig. 99 The dashboard includes pull request statistics and a link to view more details.

Enable CodeScene’s Status Badges

At this point you might want to add CodeScene’s status badges to your project as well. Status badges allow your teams to keep an eye on the health of their projects at a glance:

Keep track of knowledge loss when developers leave the project, as well as overall code health.

Fig. 100 Keep track of knowledge loss when developers leave the project, as well as overall code health.

The status badges are intended for embedding in your GitHub README file. Sample GitHub markup is provided on the “Status Badges” configuration tab.

Constraints and Limitations

  • CodeScene’s pull request integration requires a paid plan.

  • Pull requests created from forked repositories are supported with a caveat: If the fork’s branch (from which the PR has been created) is not up to date with the corresponding upstream repository’s main branch then it’s possible that CodeScene fails to detect a code health decline (or an improvement) due to obsolete versions of changed files in the fork. This should only happen when the files modified by the PR has been changed, in the meantime, in the upstream repository . A recommended remedy is to keep forks synchronized with upstream repositories.