Integrate Automated Code Health Reviews in Pull Requests

⚠ Attention: This document describes early access features. The improved PR functionality is in Beta stage, and available for opt-in customers. Contact us if you want early access.

CodeScene’s Pull Request integration is your team’s automated defense against technical debt while aligning with your organization’s goals. The integration offers an automated Code Health review, ensuring all new and modified code meets your quality standards. This simplifies your team’s code review process, improves the maintainability and readability of the code, and aligns with organizational goals.

Fig. 66 An example of CodeScene’s automated code review, illustrating how the PR integration can serve to support reviews and act as quality gates.

Key Benefits

• Fully automated Code Health review: The review is fully automated – the CodeScene tool creates the necessary webhooks, and reviews any opened or modified Pull/Merge Requests.

• Supports all major Git hosting platforms: The review works for GitHub, BitBucket, GitLab, Azure, and Gerrit (on-prem CodeScene only).

• Act as a quality bar: Avoid taking on new technical debt by enforcing Code Health standards on both new and existing code.

• Adapts to your organizational goals: Specify the code quality profile that matches your context, ambitions, and goals. Start with a minimum safety-net, enable deeper quality gates as developers become accustomed to the feedback. The custom code quality profiles let you chose the level that is right for you.

• Support Code Health decisions and improvements from technical leaders to the team: The automated code review lets you monitor and act on any Goals you specify in the CodeScene UI. Decided to refactor a piece of code? The PR review lets you know if the goal is being met.

• Understands that one size does NOT fit all: CodeScene lets you tailor how the reviews work. For example, you can chose different quality bars and gates for different portions of the codebase. Or, you could enable the PR reviews for just one team, leaving other teams unaffected. (This is really handy if you have a monolithic codebase with many teams). These more advanced configuration options are available via a config-as-code concept described in this document.

Enable the PR Integration by selecting a Quality profile

Depending on your scope, you have two options for enabling the PR integration:

1. Global configuration: Specify a shared, global configuration which is applied across all CodeScene projects.

2. Project-specific configuration: You can also enable the PR integration per project. If you have a global configuration, then the project-specific configuration will override it.

When enabling the PR integration you need to select a code quality profile. Here are your options:

Fig. 67 Select the code quality profile that marches your context and ambitions.

NOTE: As you see, there are two choices: either select a pre-defined quality profile, or go for the more flexible option of configuration as code. This document covers both options, as well as guidance on when to use one over the other. (Spoiler: go with the config as code if you need different rules for different parts of the code). Before we get to that, let’s start by explaining how the quality profiles work.

What are the quality profiles used in a PR review?

The purpose of the profiles is to adapt CodeScene to your organizational context and ambitions. That way, you align the low-level gates seen by developers doing a PR with a higher organizational purposes. (e.g. “Remediate technical debt”, “Maintain the current code quality level”). The following profiles are pre-defined:

• The Bare Minimum: This is the recommended starting point. The profile enables all checks in the “Minimal Safety Net” category. It enforces all critical Code Health rules. Critical Code Health rules are the ones you never want to violation: Low Cohesion, Deep Nested Logic, Bumpy Road, Brain/God Functions etc. In addition, this profile ensures that new code doesn’t introduce technical debt, and prevents any Code Health decline in Hotspots.

• Clean Code Collective: In addition to the “Minimal safety net”, this profile also enables all checks within the “Quality Guardians”. The main difference is that advisory Code Health rules are enforced, too. An advisory rule is more opinionated. (But it’s an opinion backed by published research). Using this profile, you get the most out of CodeScene as our checks will fail for any file where the code health declines. Note that this option matches the Code Health rules used in the Code Red paper .

• Pay down tech debt: This is the strongest profile as it enables all checks. Specifically, the “Elevate Code Health” category will start to enforce the Goals you specify in the CodeScene UI. (Examples include “Supervise” and “Refactoring” goals). The profile is a good fit if you look to really optimize your development productivity and are prepared to invest in technical debt remediations.

• Customizable Safeguards: Chose this option if you want to cherry pick checks across the various categories. For example, you might want to combine the “Minimal Safety Net” with targeted Code Health improvements indicated by the goals in the “Elevate Code Health” category.

Getting Started: Select a Profile Matching Ambition and Context

The quality profiles aren’t intended to be static. Rather, you’ll find that you might start with one profile but soon adopt more rules and checks over time. This section captures the core use cases: pick the situation which matches yours, and run with its recommendations.

Getting started with CodeScene on an existing codebase

Start with the The Bare Minimum profile. This profile offers a more gentle onboarding path to developers just getting accustomed to CodeScene as it doesn’t enforce advisory Code Health rules. Once your dev team is successfully onboarded, we recommend that you elevate your profile to the Clean Code Collective.

Enabling PRs in a codebase with Technical Debt

Our recommended choice depends on the maturity of the organization: if you have effective development practices in place (e.g. small and focused PRs), then you should go for the Pay down tech debt profile.

If on the hand you happen to have low Code Health (less than 5.0), and might struggle with large and sweeping changes (PRs with 30+ files), then we recommend a Customizable Safeguard: start with the categories “Minimal Safety Net” and “Elevate Code Health”. Specify your selected refactoring targets via Goals in the CodeScene UI, and have the autoamted PR review support you on that journey.

Enabling PRs in a Healthy codebase

If your Code Health is in the green and healthy space, we recommend using the Clean Code Collective profile. That guarantess your code stays healthy for product iterations to come.

Starting with CodeScene greenfield project

Apply the Clean Code Collective profile – there’s really no excuse not to.

Custom Quality Gates: Advanced Config as Code

The advanced configuration allows your organization to apply different checks for different portions of the code, or even use different gates for different teams: you get fine-grained control over where and how the quality gates are activated. (Contrast this to the pre-defined quality profiles which apply the same rules to all code inside the CodeScene project).

Use the advanced config in the following situations:

• Limit the gating scope: You have a monorepo where you might want to enable the PR reviews for one team, but not the others.

• Apply different rules for different parts of the codebase: Maybe you have some legacy components where the bare minimum profile is enough, but other parts of the code – eg. new components and services – require a higher quality standard.

• A gentle onboarding path: Support an iterative CodeScene rollout where the PR reviews are enabled only for specific parts of the code or specific teams without impacting people working in other parts of the code.

• Configuration as code: You want to track configuration changes and be able to rollback to a previous configuration.

How to specify the advanced configuration

The configuration is specified in a JSON file and checked into version-control inside the Git repository where the config applies. The file must be placed under a .codescene/ directory in the repository root. The name of the JSON file is .codescene/custom-quality-gates.json.

Grab the template file from the CodeScene UI on the configuration page for the PR Integration. Tailor the content of the template to your needs. The rest of this section walks you through the various options: filter by team, filter by content path, combine multiple quality profiles, or tailor the low-level gates. Let’s look at an initial example:

{
  "usage": "Persist this file inside your repositories as .codescene/custom-quality-gates.json",
  "rule_sets": [
    {
      "quality_gates": "bare_minimum",
      "quality_gates_doc": "Here we specify a quality profile. The options are: bare_minimum, clean_code_collective, pay_down_tech_debt.",
      "matching_content_path": "src/Server/**",
      "matching_content_path_doc": "Specify a glob pattern relative to this repo root."
    }, {
      "quality_gates": "clean_code_collective",
      "quality_gates_doc": "Let's use a stricter profile for the rest of the code.",
      "matching_content_path": "*/**",
      "matching_content_path_doc": "Apply this rule to all code outside of src/Server."
    }]
}

The preceding JSON file illustrates how the matching_content_path filter lets you apply different quality profiles to different parts of the codebase: the src/Server component will get the “Bare Minimum” profile, while all other code must adhere to the more strict “Clean Code Collective” which includes advisory Code Health rules too.

NOTE: The matching_content_path is relative from the Git repository root. So in the preceding example, the src/Server expects the repository to contain a top-level src folder.

Another option is to specify a filter based on development teams rather than content paths. Here’s how the example above would look:

{
  "usage": "Persist this file inside your repositories as .codescene/custom-quality-gates.json",
  "rule_sets": [
    {
      "quality_gates": "bare_minimum",
      "quality_gates_doc": "Here we specify a quality profile. The options are: bare_minimum, clean_code_collective, pay_down_tech_debt.",
      "matching_team": "Backend Team",
      "matching_team_doc": "Specify a team name: the gates are activated each time a member of that team modifies code."
    }, {
      "quality_gates": "clean_code_collective",
      "quality_gates_doc": "Let's use a stricter profile for all other teams.",
      "matching_content_path": "*/**",
      "matching_content_path_doc": "Apply this rule to all other code changes."
    }]
}

The preceding JSON file applies the matching_team filter to define custom rules for a fictitious “Backend Team”. Each time a member of that team modifies some code – no matter where that code is located in the repository – the corresponding quality gate gets triggered. Also note that the JSON file keeps the more strict profile for all other code. That is, if a member of any other team modifies the code, then the “Clean Code Collective” profile will be applied.

In addition, the JSON format also allows you to customize combinations of the low-level gates. Here’s an example where we the advanced config lets you create a customized quality profile targetting Hotspots only:

{
  "usage": "Persist this file inside your repositories as .codescene/custom-quality-gates.json",
  "rule_sets": [
    {
      "quality_gates": {
        "hotspot_decline": true,
        "critical_health_rules": false,
        "refactoring_goals": false,
        "supervise_goals": false,
        "new_code_health": false,
        "advisory_health_rules": false,
        "codeowners_for_critical_code": false
      },
      "quality_gates_doc": "We only apply the gates to Hotspots inside the Server component; no other gates.",
      "matching_content_path": "src/Server/**",
      "matching_content_path_doc": "Specify a glob pattern relative to this repo root."
    }, {
      "quality_gates": "clean_code_collective",
      "quality_gates_doc": "Let's use a stricter profile for all other code.",
      "matching_content_path": "*/**",
      "matching_content_path_doc": "Apply this rule to all other code changes."
    }]
}

The preceding JSON file offers a (very) releaxed quality standard for anything inside the src/Server component: only Hotspot files are being guarded. (This could be an example of a low-friction, initial gating strategy when adopting CodeScene to legacy code). Also note that we provide the same catch-all mechanism as before for other code to ensure it meets higher quality standards.

Finally, let’s look at a common use case. Say that you have a mono-repo with 200 engineers working on it. One team with, say, 10 developers want to enable the PR reviews, yet wait with enabling them for the other devs who might not be familiar with CodeScene yet. The following config solves the problem by providing a final catch-all rule that disables all gates:

{
  "usage": "Persist this file inside your repositories as .codescene/custom-quality-gates.json",
  "rule_sets": [
    {
      "quality_gates": "clean_code_collective",
      "quality_gates_doc": "Our part of the code use a strict ruleset",
      "matching_content_path": "src/Client/**",
      "matching_content_path_doc": "Only enable the strict gates to the Client component."
    }, {
      "quality_gates": {
        "hotspot_decline": false,
        "critical_health_rules": false,
        "refactoring_goals": false,
        "supervise_goals": false,
        "new_code_health": false,
        "advisory_health_rules": false,
        "codeowners_for_critical_code": false
      },
      "quality_gates_doc": "Disable the gates for all other code.",
      "matching_content_path": "*/**",
      "matching_content_path_doc": "This is the catch-all glob pattern."
    }]
}

In which order are the advanced configuration options applied?

• Rules are tried in order and the first rule that matches is applied.

• If no rule is matched, all quality gates are enabled.

• If you want to use a different default set of gates, then specify a rule with "matching_content_path" : "**/*" as the last rule in the JSON file. For example, you could choose to have all gates disabled by default. (This use case is handy in case you roll out CodeScene in a monorepo and only want to apply the PR review on parts of the codebase).

• The PR review always uses the .codescene/custom-quality-gates.json available on the branch. So if you make changes to the configuration on the main branch, you need to rebase any existing development branches before the profile changes are applied.

Usage Tips: Suppress Review Findings, Follow-Up on the Actions, and get Status Badges for your Code

CodeScene’s automated code reviews are based on years of research and experience in building productive teams. However, being a tool, there might be situations where you disagree with the findings or, you might simply find that a pressing deadline is more important that acting on the PR info. Both situations are OK. The important thing is just that each Code Health decline is a conscious decision, and not just something that “happens”.

To get the most out of the tool, use CodeScene’s suppression mechanism documented in the next section.

Suppress Pull Request Findings

You can suppress individual findings in a PR check. The suppression affects the PR check and subsequent checks on the same pull request.

The option is available on the results page in CodeScene. You can get to this page from PR check via the results link.

Fig. 68 Add a suppression of a finding for this PR

Pull Requests: Statistics, Actions, and Impact

Available in Analysis Results, System menu, PR Statistics

No matter what baseline we start from, we never want our code to become harder to read, understand, or maintain. CodeScene’s pull request integration lets you stay on top of your development so that you can prioritize and act upon any negative trends. For that purpose, CodeScene visualizes the impact of pull requests over time:

Fig. 69 View the impact and developer action due to the pull request integration.

The presented statistics are:

• Checks performed: This is the total number of pull request analyses run by CodeScene.

• Findings Detected: A finding is either a violated goal (e.g. a planned refactoring never happened), a reported code health decline, or new code with code health findings.

• Findings Fixed: A number of detected findings that are acted upon and mitigated.

• Findings Ignored: This is the number of detected findings that were merged as-is, without remediation, to the main branch. Ignored findings lead to a failed goal and/or a code health decline.

• Findings Suppressed: This is the number of findings that are suppressed

You can also inspect the details of individual pull requests if you need to drill down even further:

Fig. 70 CodeScene presents detailed statistics per pull request, allowing a deeper drill down.

You can find the pull request statistics on your project’s analysis dashboard:

Fig. 71 The dashboard includes pull request statistics and a link to view more details.

Enable CodeScene’s Status Badges

Available in Project Configuration, Status Badges.

Status badges allow your teams to keep an eye on the health of their projects at a glance:

Fig. 72 Keep track of knowledge loss when developers leave the project, as well as average code health.

The status badges are intended for embedding in your GitHub README file. Sample GitHub markup is provided on the “Status Badges” configuration tab.