DevSensei: PR Workflow Automation

What is DevSensei?

Table of Contents

• Getting Started 🔰

• Workflows 🔰

  • How does it work? 🔰

  • Conditions 🔰

  • Actions 🔰

    • When Do Actions Run 🔰🔬

    • Run Actions More Often 🔬

  • Merge checks 🔰

    • Example

  • Syntax support in IDE for DevSensei configuration 🔰

    • IntelliJ IDEA & JetBrains IDE’s

    • VS Code

• Migration: From Code Owners to DevSensei 🔰

  • What advantages has DevSensei compared to Code Owners?

  • How can I migrate from Code Owners to DevSensei?

  • Code Owners Settings NOT supported by DevSensei equivalents 🔬

• Reference: devsensei.yaml

  • Workflows 🔰

    • Properties

  • Conditions 🔰

    • Condition evaluation

    • Condition attributes

    • Aggregate condition operators 🔬

  • Custom Attributes 🔰

    • Custom Groups

    • Syntax

    • Code Owners

    • Syntax

  • Expressions 🔬

    • Expression types

    • Expression Operators

    • Properties 🔬

    • Lambda functions 🔬

    • Syntax reference

  • Retrigger On 🔬

    • Retrigger Expressions

  • Actions 🔰

    • add-reviewers

    • add-watchers

    • unapprove

    • add-comment

    • schedule-auto-merge

    • add-codeowners

    • cancel-auto-merge (Planned)

  • Merge Checks 🔰

    • Custom Check

    • Alias: Minimum Approvals

    • Alias: Minimum Successful Builds

    • Alias: No Open Tasks

    • Code Owners Merge Check

    • Merge Check Use Cases

  • Share Configuration with Includes 🔬

    • Sharing Parts of the Configuration with YAML Anchors 🔬

  • Configuration-Changes and Existing Pull Requests 🔬

DevSensei is a pull request workflow automation solution integrated into the Code Owners App.

It is for teams who want to:

• reduce the overhead of working with pull requests.

  • e.g. auto merge minor changes after the automated checks passed

• automate workflows in pull requests.

  • e.g. assign the right reviewers

• follow and enforce team policies.

  • e.g. add tasks to the pull request based on what changes are made, and block merging until the tasks are completed.

Getting Started 🔰

1. Enable DevSensei for your repository:

   1. Navigate to the repository → Repository Setting (⚙️ on the left) → DevSensei | Code Owners Settings. Alternatively, the setting is also available at the project level.

   2. In the settings panel, enable the DevSensei Workflows

2. Add the sample devsensei.yaml file below to the default branch of your repository.

Workflow to add comment on pull requests ready to review (basic example)

3. Done! When you create a pull request, then the comment will be added. On draft pull requests the comment is added as soon as the pull request is marked as ready to review.

Some more examples

Workflows 🔰

Each workflow automates specific tasks for the pull requests of your team.

A workflow includes:

1. a unique name to identify the workflow

2. a set of conditions to be met. Once these conditions are met the actions are executed

3. optionally: a set of actions to be executed when the conditions are met for a pull request

4. optionally: a set of merge-checks (active when workflow conditions are met) that will prevent merging unless they succeed.

How does it work? 🔰

Configure the workflows for your team

1. on the default branch

2. in the devsensei.yaml file

3. at the top-level directory of your repository.

The devsensei.yaml file consists of a set of workflows.

Conditions 🔰

Use conditions to decide for which pull requests to run your actions, or enforce extra merge checks. Conditions give you full control to tailor a workflow to run the actions exactly when you need.

A basic condition is a comparison on a pull request attributes like title, source and destination branch. Then compare the attributes with an operators like equality =, glob matching ~=, negations not to the desired value.

All conditions in the conditions list must be fulfilled to run an action. Use or and and blocks if you need logical combinations of conditions.

Define complex conditions using expressions.

Actions 🔰

Actions do things for you, like adding comments and/or reviewers, merging the PR, etc.

A workflow has one or more actions.

When Do Actions Run 🔰🔬

Actions run when the conditions change from false to true.

Then, actions do not run again as long as the condition stays true.

When conditions go again back to false and then true, the actions run again.

This concept is called Edge Triggering, as actions are triggered on the 'edge' of the signal when the conditions do change.

Run Actions More Often 🔬

Sometimes you need to run the actions of a workflow more often, for example every time new commits are made to the pull request.

For that, there is the retrigger-on section. If the value of one of the attributes in the retrigger-on section changes, it will "reset" the condition signal and if the conditions are currently met, a new edge trigger happens, and the workflow’s actions will run again.

Examples:

Merge checks 🔰

Merge checks let you define extra rules, enforced while workflows are active, to protect your team’s repository from merging invalid PRs. For example, only accepting PR’s that follow your specific branch naming conventions.

Compared to Bitbucket’s own builtin merge checks, DevSensei allows you to use the same expressions and attributes as workflow conditions to customize exactly when checks should prevent a merge.

Example

Bitbucket has a builtin merge check to enforce a minimum number of approvals. However, the only configuration is a number. With DevSensei you can scope this further by, for example, restricting the check to specific branch patterns:

Syntax support in IDE for DevSensei configuration 🔰

The app provides a YAML Schema for the devsensei.yaml file.

Benefits:

• Auto-completion of YAML keys

• Basic validations, (e.g. ensure that a workflow has actions)

• Documentation of YAML elements within the editor

• Showing code examples from the Spec as help

Download the YAML Schema from Bitbucket:

https://YOUR_BITBUCKET/rest/codeowners/1.0/devsensei/schema

Depending on your IDE, map that YAML Schema to files named devsensei.yaml.

IntelliJ IDEA & JetBrains IDE’s

1. Go to IntelliJ IDEA (or other Jetbrains IDE) settings

2. Search for JSON Schema Mappings

3. Add a new mapping:

   1. Name: DevSensei Schema file

   2. Schema URL: https://YOUR_BITBUCKET/rest/codeowners/1.0/devsensei/schema

   3. Schema version: JSON Schema version 7

   4. File: devsensei.yaml

VS Code

VS Code with the RedHat YAML plugin can either use an inline reference to the JSON Schema

or have a global mapping within the VS Code settings.json:

Migration: From Code Owners to DevSensei 🔰

What advantages has DevSensei compared to Code Owners?

• devsensei.yaml can share common rules across repositories with included devsensei.yaml files called "Includes" to reduce duplication and maintenance efforts.

• Common configuration parts (e.g. reusing the list of reviewers) can be shared with YAML anchors.

• DevSensei allows to build the automation you want with combining conditions and actions.

• DevSensei currently supports add-reviewers, unapprove, codeowners-check (and other merge-checks), add-comment, and schedule-auto-merge. We will add more actions in the future to automate your pull request workflow. Let us know what actions you are looking for.

• devsensei.yaml is read from the default branch of your repository. This will reduce the maintenance efforts significantly when the automation for the repository needs changes.

How can I migrate from Code Owners to DevSensei?

To start using DevSensei from your existing CODEOWNERS file, you have two options:

1. a) Automated migration: migrate your CODEOWNERS file to devsensei.yaml with the built-in migration support (see button Download generated devsensei.yaml).

2. b) Manual migration: migrate your CODEOWNERS settings to their equivalents in devsensei.yaml.

   1. Copy the assignment rules from the CODEOWNERS file to the rules section of codeowners custom attribute in devsensei.yaml.

   2. For the settings, use a combination of workflow conditions, and options for the add-reviewers, add-comment, and unapprove actions (see the Actions paragraph below).

   3. For the custom Code Owner groups (e.g. @@@my-group @peter @anna), add a custom-groups custom attribute to the workflow.

   4. For the merge checks, add a codeowners-check to the merge-checks section of a workflow.

3. push the devsensei.yaml file to the root directory in the default branch of your repository

4. enable DevSensei under repository settings -> DevSensei | Code Owners -> DevSensei Workflows -> Enabled

5. when the app sees a devsensei.yaml file, it will use that instead of CODEOWNERS.

Below you can see both a CODEOWNERS file and the equivalent devsensei.yaml file. This should help you to migrate from your Code Owners rules to the new YAML format.

The format of the Code Owners rules is the same, so you can copy that to the rules section of the codeowners custom attribute.

CODEOWNERS

devsensei.yaml

Code Owners Settings NOT supported by DevSensei equivalents 🔬

Reference: devsensei.yaml

Workflows 🔰

Each workflow is meant to automate specific tasks for the pull requests of your team.

Properties

Conditions 🔰

The conditions property of a workflow is configured with a list of conditions.

Condition evaluation

Each condition is a boolean expression, so composition with and and or obeys the usual rules for boolean algebra.

If there are no conditions (i.e. there is no mapping, or an empty list), then the condition is always met by default, so any action in the workflow run at least once. This means that each additional condition further constrains the cases where actions should be applied. If there are more than one condition in a workflow’s conditions property, then they must be all true simultaneously for the overall condition to be met (i.e. they are combined with and implicitly).

Conditions are dynamic expressions, and can retrieve and compare metadata (known as attributes) associated with the current pull request. Conditions are evaluated when a pull request is created, whenever the pull request or its various metadata changes.

Condition attributes

These are the various attributes available to use in conditions, retrigger-on, and merge-checks. Each attribute results in a typed value when evaluated.

Aggregate condition operators 🔬

There are two logical operators that can compose a list of conditions

Custom Attributes 🔰

Custom attributes are configurable attributes to be used within a workflow, for example as part of the configuration of an action or a merge check.

Available custom attributes are:

• custom-groups

• codeowners

Custom Groups

Custom groups define a set of users grouped together as a single entity. There are similar to Bitbucket or Reviewer user groups, but they are only defined within a workflow.

A group is defined by a name and a list of identifiers. An identifier can be

• a reference to a user (as individual via email or username) (e.g., @alice or alice@domain.com), or

• a reference to another group (custom group, Bitbucket group or reviewers group) using @@<group-name>.

Syntax

Custom groups are defined as a yaml mapping from the name of the group to the list of identifiers within that group.

Example:

Code Owners

The codeowners custom attributes is defined by a rules member, configuring what files are owned by which people.

Syntax

The rules are defined using a multi-line string where each line follows one of the two following patterns:

• <file pattern> <code owners>: to assign files to some code owners

• !<file pattern>: to exclude files from being assigned.

(Rule line order matters: the last matching line wins.)

See our complete reference for more context and information.

Example:

Expressions 🔬

Expressions are small computations on Pull Request attributes. Expressions are most useful in workflow conditions and custom merge checks. Expressions can be as simple as a constant (like a number or a string) or an attribute value, but expressions can also be more complex manipulation of attributes.

Some examples of expressions are

• draft

• true

• 'Hello World'

• source ~= 'bugfix/*'

• count(approved-by)

• builds.successful

• forall(changed-files, $1 ~= '**/*.js')

Expressions can be composed in any form or fashion using operators described below, as long as type requirements are fulfilled. If an expression does not type-check (e.g., source > 3), it will make the devsensei.yaml file invalid.

Expression types

Here are the following types that can be involved in expressions

Expression Operators

Use Operators to transform expressions into more complex ones. For example, to compare two int values or to count the number of elements in a set. Operators can be infix (only for binary operators), prefix or call. Infix operators are placed within their arguments, prefix operators must be placed before their arguments, and call operators must enclose their argument(s) in parenthesis, separated by commas.

Properties 🔬

Some attributes have properties with a sub-set of the data.

Syntax

<attribute>.<property>

Applying a property will access the property of the attribute.

Lambda functions 🔬

Some functions require to have a lambda function as argument, for example exists or forall. A lambda function can be composed as any usual expression, where the argument of the lambda is represented by $1.

For example, the following expressions are valid lambdas:

• $1 ~= 'alice'

• !$1

• count($1) < 50

• contains(reviewers.approved, $1)

Syntax reference

Expressions can be

• a simple boolean, true or false

• an integer value, e.g., 7, -6, +2

• a string literal, enclosed in single quotes, e.g., 'Hello world'. Within quotes, single quotes ' and backslash \ must be escaped by preceding them with a backslash.

• an attribute value, e.g., source, approved-by

• of the form <expression> <infix operator> <expression>, e.g., source ~= 'bugfix/*'

• of the form <prefix operator> <expression>, e.g., !draft

• of the form <call operator>(<expression> [, <expression>]*), e.g., count(open-tasks), contains(reviewers, 'user')

• of the form <expression>.<property>, e.g., changed-files.modified

• of the form ( <expression> ), e.g. (changed-files ~= '**/*.js') = true

Outside of string literals, spaces are insignificant.

Retrigger On 🔬

The retrigger-on section is optional. If configured, it defines a set of expressions that are continuously evaluated, similar to conditions.

Example

It provides more fine-grained control over when edge-triggering occurs. Normally, actions only run on the "edge" when the condition changes from "not-met" to "met". For example, if attribute values change, but the condition is still met, then normally actions will not run again.

However, if retrigger-on is configured, then whenever one of its expression values changes while the condition is met, the signal for the condition will temporarily flip, enabling a new edge-trigger even though the condition is already met.

You can consider the behavior similar to two electrical signals: condition (A), and retrigger-on (B), combined with a logic gate (A & !B). Signal B is only activated in the instant where a change occurs, but otherwise deactivates, which creates more "edges" in the combined signal.

Concept of retrigger-on

Without the retrigger-on section the workflow actions would only run twice. The effect of retrigger-on makes it so that the workflow actions actually run 4 times instead.

Retrigger Expressions

Concretely, in a single expression you can either specify a watchable attribute, or observe changes to the combined diff.

Look at the attributes table to see which are watchable. Let us know if you would like more kinds of checks to be supported in this section.

Actions 🔰

List of available actions that fuel your automation.

Let us know if you miss an action.

add-reviewers

Suggests and/or adds reviewers to a pull request. (See also add-watchers for watchers instead)

add-watchers

Adds watchers to a pull request. (See also add-reviewers for reviewers instead)

unapprove

Unapprove all or some of the approved reviews in the Pull Request.

add-comment

Adds a single comment or task to a pull request. Can either be custom content, or automated reporting of action results.

schedule-auto-merge

Schedules this pull request to be automatically merged once all merge checks are clear.

Prerequisites

This action requires:

• Bitbucket 8.15 or newer

• The Bitbucket Auto-Merge feature must be enabled for the repository

Auto-Merge Behavior

When the action runs, it submits an Auto-Merge for this pull request. Bitbucket then will merge the pull request, depending on its state:

Check the Bitbucket documentation for more details on how auto merge behaves.

Because the Auto-Merge request may be cancelled by new commits, you may decide to ask again for auto-merging. See the Retrigger On section for more details

Auto-Merge Use Cases

The schedule-auto-merge action makes it simpler to automatically accept lower risk pull requests by merging quickly. As it is based on Bitbucket’s Auto Merge feature, after scheduling, Bitbucket will still prevent merge if there is a veto from one of the merge checks enabled for your repository (for example anything in the merge-checks section).

Other Auto-Merge Use Cases (Planned)

The following ideas rely upon planned or otherwise unimplemented features. Contact support to let us know your use case.

• Schedule merge when a no reviewer requested changes. (requires new attribute)

• Wait until another PR has been merged (e.g. merge queue)

• Check that JIRA ticket is attached before merge (requires a new attribute)

• Check total size of change is within a small range (requires a new attribute).

add-codeowners

Adds Code Owners as reviewers to a pull request.

cancel-auto-merge (Planned)

Cancels any pending auto merge. This action is planned. Contact support to let us know your use case.

Merge Checks 🔰

List of enforced checks that protect against merging unless they all succeed.

Let us know if your use case is not covered.

Merge checks reuse the same expressions as conditions, and can be grouped with a custom title. DevSensei also offers alias merge checks for common use cases.

Custom Check

Example

Properties

Alias: Minimum Approvals

Equivalent to a custom merge check with the single condition count(approved-by) >= n for positive number n.

Example

Properties

Alias: Minimum Successful Builds

Equivalent to a custom merge check with the conditions:

• count(builds.successful) >= n for a positive number n.

• count(builds.failed) = 0 disallow failed builds

• count(builds.in-progress) = 0 wait for all builds to be complete

Example

Properties

Alias: No Open Tasks

Equivalent to a custom merge check with the single condition count(open-tasks) = 0.

Add this merge check by adding the string no-open-tasks to the workflow merge-checks list.

Example

Properties

This merge check alias has no property.

Code Owners Merge Check

This merge check enforces rules as described in the Code Owners documentation. For existing CODEOWNERS users, copy your merge checks to this section.

Properties

Example

Merge Check Use Cases

The merge-checks section gives more control over which pull requests should be merged, for example applying unique checks to specific branch patterns.

Planned Use Cases

The following ideas rely upon planned or otherwise unimplemented features. Contact support to let us know your use case.

Fine-grained equivalents to built-in Bitbucket merge checks

Supporting the following would allow to create finer grained checks (e.g. by filtering on destination branch pattern):

• Require no "requested changes" status

Support non-functional requirements

• Require that a pull request has been open long enough to be seen (time-based condition)

• Prevent merge to a release branch on a specific day.

More fine-grained attributes

• check text contents of open-tasks, e.g. if your team uses a marker to indicate some tasks can be ignored.

• check specific user in approved-by.

Code Owners merge checks

Over time we hope to migrate the merge checks from the add-codeowners action, starting with the codeowners-check option. We could possibly make them more flexible, for example check the intersection of approved-by with the added code owners.

Share Configuration with Includes 🔬

With includes, common Devsensei workflows can be shared both across repositories as well within projects in a monorepo.

To include a devsensei.yaml file from another repository, use this syntax:

Only repositories of the same Bitbucket project are supported.

The Bitbucket users require READ access to the repository with the shared configuration. Otherwise users will get permission issues, because they cannot access the required configuration.

When a plain YAML file path is specified, the included file is read from the same repository:

• The files are read from the latest commit of the default branch

• Only one level of includes are support (no recursion supported)

Sharing Parts of the Configuration with YAML Anchors 🔬

Use YAML Anchors to define reusable content in the same YAML file.

Example showing how to share conditions across workflows:

• Anchors can be used for any content. For example, to define reusable custom user groups, or conditions.

• Reference must reference anchors in the same file. It is not possible to define an anchor in one file, and reference it from another file.

• Anchor names cannot contain the [, ], {, }, and , characters.

• Anchors can be defined anywhere in the file, although we encourage to use the shared section at the top of the file for clarity and easier maintenance.

Configuration-Changes and Existing Pull Requests 🔬

DevSensei reads the configuration always from the default branch. That means that all open pull requests will use the same configuration and changes in the configuration affect all pull requests.

When you change the configuration, then when DevSensei runs the next time, the new configuration is applied.

Table 1. The next time DevSensei runs: