Bash auto-completion helps with the burden of remembering all the different tasks for the various sub-projects in a build. It also provides new ways to discover the features of a build in an exploratory way. In the following, I give a description on how to implement bash auto-completion, based on the following requirements:

• Fast auto-completion in the sub-second range. To compute the auto-completable suggestions, the Gradle project needs to be fully configured, which takes at least a few seconds for medium-sized builds. This means that the list of auto-completable suggestions needs to be precomputed.
• The auto-completable suggestions to show should be customizable. Some tasks, even if available on the project, should perhaps not be shown. This might be tasks that are not relevant for the developer but used in later integration stages. Other parameters to the build might be interesting to show in addition to the restricted set of tasks. This might include other parameters that can be passed to the Gradle build, such as task parameters or options to set the Gradle output logging level.
• No change to the Gradle wrapper scripts are needed.

The solution presented is two-fold. First, we introduce the shorthand gr to run the Gradle wrapper from any directory of the multi-project build. Auto-completion is then implemented in the second step. Note that, although Bash is a Unix shell, the following also works for Bash ports on Windows such as MSYS (part for example of msysGit).

Gradle shorthand gr

There are different design choices to implement the shorthand gr, influenced by the features it should have. The implementation given here works as follows. It searches from the current directory upward until it finds a Gradle wrapper script which it then invokes.

We first implement the function upsearch that can search upward in a directory tree. All of the following goes into the Bash startup file ~/.bashrc:

upsearch () { # adapted from https://gist.github.com/lsiden/1577473
  local the_test=$1
  local curdir=`pwd`

  while [[ "`pwd`" != '/' ]]; do
    if eval "[[ $the_test ]]"; then
      pwd
      cd "$curdir"
      return 0
    fi
    cd ..
  done
  cd "$curdir"
  return 1
}

We then implement a function _findgradledir that searches for the Gradle wrapper file named gradlew:

_findgradledir () {
  upsearch '-f gradlew'
}

The function _gradlew, implemented in the following, uses _findgradledir to find the wrapper file, which it then invokes.

_gradlew () {
  local gradle_project_dir=`_findgradledir`
  if [ -z "$gradle_project_dir" ]; then
    echo "Gradle project directory not found" 1>&2
    return 1
  fi
  "$gradle_project_dir/gradlew" -p "$gradle_project_dir" "$@"
}

Last, we define the command gr. Its implementation is directly given by the function _gradlew. We can also pass default parameters to the wrapper invocation. For example, to improve startup and execution time of the Gradle invocation, we add the --daemon flag that activates the Gradle daemon:

alias gr="_gradlew --daemon"

Auto-completion

Implementing auto-completion requires not only some bash scripting but also some implementation on the Gradle side that gives us the list of auto-completable suggestions. We introduce a task named bashCompletion that generates the list of tasks in a multi-project build and writes them line-by-line into a text file. The bashCompletion task is defined in the root Gradle project (but could be put for example in an init script if it were to be used for many builds):

task bashCompletion << {
  def allTaskNames = []
  getAllTasks(true).each { proj, taskSet ->
    taskSet.each {
      if (proj.parent) {
        allTaskNames += proj.name + ':' + it.name
      } else {
        allTaskNames += it.name
      }
    }
  }
  def commands = (defaultGradleCommands + allTaskNames).sort().unique().join('\n')
  file('gradle-autocomplete').text = commands
}

The presented implementation is a very basic one and many customizations are possible here. For example, task exclusion lists or other build parameters could be used. The auto-completable suggestions are stored sorted in a line-based text file so that it has a nice format for diffs when we put it into version control. More information on why this might be useful is provided in another blog article here.

On the Bash side (~/.bashrc), basic auto-completion is fairly trivial to implement. We use the previously defined function _findgradledir to find the text file of auto-completable suggestions created by our bashCompletion task:

_gradle_complete()
{
  local cur
  _get_comp_words_by_ref -n : cur

  local gradle_project_dir=`_findgradledir`
  if [ -z "$gradle_project_dir" ]; then
    return 1
  fi
  
  if [ ! -f "$gradle_project_dir/gradle-autocomplete" ]; then
    return 1
  fi
  
  local commands=$(cat "$gradle_project_dir/gradle-autocomplete" | tr '\n' ' ')
  COMPREPLY=( $(compgen -W "$commands" -- $cur) )
  __ltrim_colon_completions "$cur"
}

The line _get_comp_words_by_ref -n : cur is needed for bash completion to work with colons in task calls of sub-projects.

Finally we use the bash complete command to attach the implemented bash completion function to our Gradle shorthand gr:

complete -F _gradle_complete gr

As usual, the given solution is based on a specific set of requirements:

• Gradle and Eclipse build configurations should be kept in sync. Consistency between IDE settings and build system ensures that the development environment is as close as possible to the final software build.

• The overhead of keeping things consistent should be low.

• Developers should not require to install any Eclipse plugins such as the STS Gradle integration.

• Complex installation or configuration processes should not be required by the developer to get the Eclipse IDE properly set up.

• The version control system should contain all relevant information to directly start developing without doing any complicated installation or configuration. This should especially allow to evolve the infrastructure in sync with the code.

The solution I outline has two aspects:

1) Managing and generating all relevant Eclipse settings using Gradle.
2) Putting the generated Eclipse settings into the version control system.

Managing Eclipse settings using Gradle

There are numerous advantages to manage Eclipse settings using Gradle:

• Many properties can be kept in sync between Gradle and Eclipse. Such properties include source and target levels for the compiler, encoding of source files and classpath settings.
• In large multi-project builds, sharing configurations over many sub-projects can happen quite frequently. This is a task that should be easy to do. Gradle supports this nicely using its pattern of configuration as code. Common settings that are frequently used by many sub-projects can be abstracted into closures. These closures, which can be customized to one’s need, can then be applied to all relevant sub-projects. By managing the Eclipse configurations in Gradle, the same kind of customization and sharing can be done for Eclipse-related settings as well. In fact, this is not only limited to properties that are shared between Gradle and Eclipse. It is useful to manage other Eclipse configurations such as auto-save actions for the Eclipse editor or formatting rules for the Eclipse formatter. These settings can be configured on a per-project basis and do not require any changes to the Eclipse installation. It ensures that all developers submit properly formatted source-code independently on what they have configured in their Eclipse installation. It also gives them the possibility to dynamically evolve settings along with the source code without requiring any other provisioning framework.

Eclipse settings under version control

Putting Eclipse settings under version control might seem at odds with the principle of not putting derived resources into version control. One reason for this is the need of additional storage costs in the version control system. In this specific case, however, things are not that bad. Eclipse settings are stored in text files taking not much space and are line-based which is very good for diffs. Another challenge is that consistency cannot be guaranteed between sources and derived resources. This risk can be mitigated by using a Continuous Integration server which checks for consistency after each push. This is simply realized by checking for uncommitted changes in the workspace after regenerating the Eclipse files using gradle cleanEclipse eclipse. A Gradle-based implementation of this check for the Git version control system is provided at the end of the blog post.

The main reasons for why we put the Eclipse settings into version control are as follows:

• In our case, running the command gradle eclipse takes 10+ seconds (mostly due to a longer configuration-phase of the build), which is too slow as a post-checkout hook.
• It ensures that every developer is always using the correct classpath when updating or switching branches. Additionally, it does not require to regenerate the settings each time a checkout occurs. Checkouts occur way more frequently than changes to the settings!
• The developer becomes aware of unintended changes made to his Eclipse configurations (configuration file gets marked as dirty by the version control system and consistency is enforced by the integration server).

Using the given solution, it is then the responsibility of developers that make changes to the settings, for example by adding new third-party libraries, to run gradle eclipse and to check the updated Eclipse setting files into version control.

A critical requirement for the solution to work is that the generated Eclipse configurations are neither machine- nor developer-specific. In the following, I present some implementation tips that ensure this for the three Eclipse settings files that are generated when invoking the Gradle eclipse task:

.project
.setting/.org.eclipse.jdt.core.prefs
.classpath

Implementation tips

The project settings file is stored in xml format and is easily portable. It needs no further consideration. The jdt core preferences file is a Java Properties file (without the file ending .properties) and contains, among others, compilation settings. The issue with the Gradle generation of this file is that it is not deterministic. The generated properties file has as first line a commented time stamp which gets updated whenever gradle eclipse is called. To make things worse, the elements in the file are not sorted. We fix this by adding a post-action to the eclipseJdt task:

eclipseJdt << {
  def lines = outputFile.text.readLines().findAll { !it.startsWith('#') }
  outputFile.text = lines.sort().collect{ it + '\n' }.join()
}

After the settings file has been generated by eclipseJdt, we load the content of the file, remove the auto-generated time stamp which is prefixed with the comment symbol #, and finally sort the remaining options.

To ensure that the generation process of the classpath file is deterministic, we also require that the ordering of classpath items is fixed. Note that no formal guarantees regarding classpath ordering are given by the Gradle developers. What we could observe so far, however, is that the ordering is mostly deterministic with a few exceptions. By fixing these few exceptional cases, we can assume that the generation is fully deterministic (If it is not the case, we can easily detect this using our continuous integration server).

One particular exception is the use of * includes. For example, using

compile fileTree(dir: 'lib', include: '*.jar')

gives a different ordering of files across different systems. This can be fixed by sorting the dependencies using

compile files { fileTree(dir: 'lib', include: '*.jar').files.sort() }

instead.

Managed dependencies are stored in the Gradle dependency cache, which is usually located under the home directory of the current user. To get a user-independent classpath, Gradle can be instructed to replace the prefix to the Gradle cache by a named variable:

eclipse.pathVariables 'GRADLE_USER_HOME': gradle.gradleUserHomeDir

Note that this also works for the corresponding source jars, but not for Javadoc. Two things then need to be communicated to the other developers. First, the variable GRADLE_USER_HOME has to be set in the Eclipse IDE settings menu which can be accessed through Window -> Preferences -> Java -> Build Path -> Classpath Variables. To remind the developer of this, a message can be written to the console whenever the Eclipse classpath is (re-)generated.

gradle.taskGraph.whenReady { taskGraph ->
  if (!taskGraph.allTasks.findAll{ it.name == 'eclipseClasspath' }.empty) {
    gradle.buildFinished {
      println '----------------------------------------------------------------------'
      print   "Please set the Eclipse classpath variable 'GRADLE_USER_HOME' in Preferences "
      println "-> Java -> Build path -> Classpath Variable to $gradle.gradleUserHomeDir"
      println '----------------------------------------------------------------------'
    }
  }
}

Second, the interaction of the presented approach with software repositories (Maven Central or a company-local repository using software such as Nexus or Artifactory) must be understood. After checking out code, the developer has an up-to-date Eclipse classpath file. It does not ensure, however, that the developer has all the required libraries in his Gradle cache which are referenced from the Eclipse classpath file. After doing the check out, the developer might thus see a warning in Eclipse showing that some classpath items are missing. In that case, the developer simply re-executes gradle eclipse to regenerate the Eclipse settings (which should leave the settings files unchanged) and as side-effect downloads the missing libraries.

Non-managed dependencies also need special care. As Jar files are sometimes stored directly in Git along with their projects, we have to ensure that relative paths to these files are used to guarantee a user-independent classpath. The Gradle eclipse plugin unfortunately generates absolute paths. We fix this in the following way:

eclipse.classpath.file.whenMerged { classpath ->
  classpath.entries.each { entry ->
    if (entry.kind == 'lib') {
      // for jar files referenced in project folder, use relative path
      def prefix = projectDir.absolutePath.replace('\\', '/')
      entry.path = entry.path.replace(prefix, "/$eclipse.project.name")
    }
  }
}

For Windows compatibility reasons, we have to normalize the project path by replacing backward by forward slashes. The code is best illustrated using an example: The classpath entry for the JUnit library located in /home/foo/bar/XYZ/lib/junit.jar of project XYZ is converted to /XYZ/lib/junit.jar (which is the relative path format understood by Eclipse).

One last thing: To ensure that the generated setting files are kept in sync with their definitions in Gradle, a task can be created that checks for uncommitted changes whenever the files are recreated. The Java implementation of Git called JGit makes this a simple and portable task:

buildscript {
  repositories {
    mavenCentral()
  }

  dependencies {
    classpath 'org.eclipse.jgit:org.eclipse.jgit:3.6.0.201412230720-r'
  }
}

task checkUncommittedChanges << {
  def repoDir = file('.') // root of the Git repository
  def gitRepo = org.eclipse.jgit.api.Git.open(repoDir)
  def uncommittedChanges = gitRepo.status().call().getUncommittedChanges()
  gitRepo.close()
  if (uncommittedChanges.size() > 0) {
    ant.fail "Uncommitted changes check failed. Uncommitted changes found: $uncommittedChanges"
  }
}

The continuous integration server can then simply execute gradle cleanEclipse eclipse checkUncommittedChanges to check for inconsistencies.

This concludes the blog post. Future expansions into the topic might focus more on what specific Eclipse settings to manage using Gradle and what other settings to leave open for developers to customize to their needs. Any comments or suggestions are very welcome.