Uploaded image for project: 'XWiki Commons'
  1. XWiki Commons
  2. XCOMMONS-3144

Test coverage computed on Sonarcloud is not correct

    XMLWordPrintable

Details

    • Unknown
    • N/A
    • N/A

    Description

      The issue is that we execute the following for each GH repo:

              buildInternal(
                name: 'Quality',
                goals: 'clean install jacoco:report sonar:sonar',
                profiles: 'repository-snapshots,quality,legacy,coverage',
                properties: '-Dxwiki.jacoco.itDestFile=`pwd`/target/jacoco-it.exec'
              )
      

      And this means that for each maven module, the goals "clean install jacoco:report sonar:sonar" are executed. This means that the coverage report for one module is generated BEFORE the following modules in the reactor have had a chance to contribute test results for the current module, leading to lower coverage than it should be.

      To reproduce, go in xwiki-rendering (for ex), and run "mvn clean install jacoco:report -Dxwiki.jacoco.itDestFile=`pwd`/target/jacoco-it.exec -Plegacy,coverage -Dxwiki.revapi.skip=true", then open (for ex) xwiki-rendering-api/target/site/jacoco/index.html.

      Then execute "mvn jacoco:report -Dxwiki.jacoco.itDestFile=`pwd`/target/jacoco-it.exec -Plegacy,coverage -Dxwiki.revapi.skip=true" (this will regenerate the reports but from a full jacoco-it.exec file with all the module test executions in it). And reopen xwiki-rendering-api/target/site/jacoco/index.html, you'll see that the coverage is much larger (from 50+% to 80+%).

      We either need to run 2 builds in our CI pipeline (simple) or find a strategy to somehow use the jacoco:merge or jacoco:report-aggregate goals in a single reactor build. This will likely mean having a special module depending on all the other modules and gathering all the module sources to generate a full jacoco report.

      Attachments

        Activity

          People

            vmassol Vincent Massol
            vmassol Vincent Massol
            Votes:
            0 Vote for this issue
            Watchers:
            0 Start watching this issue

            Dates

              Created:
              Updated:
              Resolved: