octocov is a toolkit for collecting code metrics (code coverage, code to test ratio, test execution time and your own custom metrics).
Key features of octocov are:
- Useful both as a CI tool and as a CLI tool
- Support multiple coverage report formats.
- Support multiple code metrics.
- Support for even generating coverage report badges.
- Have a mechanism to aggregate reports from multiple repositories.
GitHub Actions for octocov is here !!
First, run test with coverage report output.
For example, in case of Go language, add -coverprofile=coverage.out option as follows
$ go test ./... -coverprofile=coverage.outAnd generete .octocov.yml to your repository.
$ octocov init
.octocov.yml is generatedAnd set up a workflow file as follows and run octocov on GitHub Actions.
# .github/workflows/ci.yml
name: Test
on:
pull_request:
jobs:
test:
runs-on: ubuntu-latest
steps:
- uses: actions/checkout@v3
- uses: actions/setup-go@v4
with:
go-version-file: go.mod
- name: Run tests with coverage report output
run: go test ./... -coverprofile=coverage.out
- uses: k1LoW/octocov-action@v1Then, octocov comment the report of the code metrics to the pull request.
It is also possible to add reports to GitHub Actions Job Summaries by editing .octocov.yml.
It can also be inserted into the body of a pull request.
Note that only pull requests from the same repository can be commented on (Reporting to GitHub Actions Job Summaries is permitted). This is because the workflow token of a forked pull request does not have write permission.
octocov acts as a code metrics viewer on the terminal.
For example, in case of Go language, add -coverprofile=coverage.out option as follows
$ go test ./... -coverprofile=coverage.outAnd run octocov ls-files , octocov view [FILE...] and octocov diff [REPORT_A] [REPORT_B]
By setting comment:, comment the reports to pull request.
# .octocov.yml
comment:
hideFooterLink: false # hide octocov linkoctocov checks for "Code Coverage" by default. If it is running on GitHub Actions, it will also measure "Test Execution Time".
If you want to measure "Code to Test Ratio", set codeToTestRatio:.
comment:
codeToTestRatio:
code:
- "**/*.go"
- "!**/*_test.go"
test:
- "**/*_test.go"By setting report: ( report.path: or report.datastores ) and diff: ( diff.path: or diff.datastores ) additionally, it is possible to show differences from previous reports as well.
comment:
updatePrevious: true
report:
datastores:
- artifact://${GITHUB_REPOSITORY}
diff:
datastores:
- artifact://${GITHUB_REPOSITORY}
The breakdown behind that comparison is folded into a Details section. Set comment.expandDetails: ( or the same key under summary: or body: ) to render it, and the Metadata of each custom metric set, already open.
When the report is stored in a GitHub Actions artifact, the coverage values of the comment link to octocov.dev, where the stored report can be browsed. Each overall coverage opens the report of the ref it describes, so the compared column opens the default branch and the current one opens the pull request, and the coverage of each file opens that file. The compared column is linked only when diff.datastores: read it out of an artifact, since that is what the page it would open serves, and no column is linked at all unless this run stores its own report in one. The file names keep pointing at the source on GitHub. Set comment.hideCoverageLink: ( or the same key under summary: or body: ) to render them as plain values again.
By setting coverage.acceptable:, the condition of acceptable coverage is specified.
If this condition is not met, the command will exit with exit status 1.
# .octocov.yml
coverage:
acceptable: 60%$ octocov
Error: code coverage is 54.9%. the condition in the `coverage.acceptable:` section is not met (`60%`)By setting codeToTestRatio.acceptable:, the condition of acceptable "Code to Test Ratio" is specified.
If this condition is not met, the command will exit with exit status 1.
# .octocov.yml
codeToTestRatio:
acceptable: 1:1.2
code:
- "**/*.go"
- "!**/*_test.go"
test:
- "**/*_test.go"$ octocov
Error: code to test ratio is 1:1.1, the condition in the `codeToTestRatio.acceptable:` section is not met (`1:1.2`)By setting testExecutionTime.acceptable:, the condition of acceptable "Test Execution Time" is specified (on GitHub Actions only) .
If this condition is not met, the command will exit with exit status 1.
# .octocov.yml
testExecutionTime:
acceptable: 1 min$ octocov
Error: test execution time is 1m15s, the condition in the `testExecutionTime.acceptable:` section is not met (`1 min`)By setting *.badge.path:, generate badges self.
# .octocov.yml
coverage:
badge:
path: docs/coverage.svg# .octocov.yml
codeToTestRatio:
badge:
path: docs/ratio.svg# .octocov.yml
testExecutionTime:
badge:
path: docs/time.svgYou can display the coverage badge without external communication by setting a link to this badge image in README.md, etc.
# mytool
  
By setting push:, git push report badges self.
# .octocov.yml
coverage:
badge:
path: docs/coverage.svg
push:By setting report:, store the reports to datastores and local path.
# .octocov.yml
report:
datastores:
- github://owner/coverages/reports
- s3://bucket/reportsThe report of the default branch is stored under the repository itself, and the report of any other ref is stored beside it under a key naming that ref.
owner/repo/report.json # the default branch
owner/repo/refs/pull/123/report.json # pull request #123
owner/repo/refs/heads/feat/x/report.json
diff.datastores: reads the report of the default branch, so the comparison shown on a pull request is against the default branch whether or not the pull requests report as well. Reporting from every run is therefore the default, and report.if: is_default_branch is only needed to keep the reports of the other refs out of a datastore, which is worth doing for github://, where each report is a commit that stays.
# .octocov.yml
report:
path: path/to/report.json- GitHub repository
- GitHub Actions Artifacts
- Amazon S3
- Google Cloud Storage (GCS)
- BigQuery
- Local
By enabling central:, octocov acts as a central repository for collecting reports ( example ).
# .octocov.yml for central mode
central:
root: . # root directory or index file path of collected coverage reports pages. default: .
reports:
datastores:
- bq://my-project/my-dataset/reports # datastore paths (URLs) where reports are stored. default: local://reports
badges:
datastores:
- local://badges # directory where badges are generated.
push: # enable self git pushWhere a report is collected from a GitHub Actions artifact ( artifact:// ), the badges of its row link to octocov.dev, and the markdown offered under Copy status badge markdown carries the same link. The pages there read a report out of the artifacts of the repository it describes, so collecting from one is what says the page is there to open. A report collected from any other datastore, and every report of a central repository running on GitHub Enterprise Server, gets plain images instead.
This example illustrates how to merge coverage reports from multiple jobs. It is useful when you are testing code branches for multiple targets (e.g. test sharding, runtime version, dependency version, OS, etc).
# .github/workflows/ci.yml
name: merge coverages from multiple jobs
on:
push:
branches:
- main
- master
pull_request:
jobs:
test:
strategy:
fail-fast: true
matrix:
shard: [1, 2, 3]
runs-on: ubuntu-latest
permissions:
contents: read # to checkout
steps:
- uses: actions/checkout@v4
- run: YOUR TEST HERE
- name: Upload coverage to workspace
uses: actions/upload-artifact@v4
with:
path: lcov.info # adjust this path to your coverage file
name: octocov-${{ matrix.shard }}
if-no-files-found: error
coverage-aggregation:
runs-on: ubuntu-latest
permissions:
contents: read # to checkout
pull-requests: write # for octocov to post comments
actions: read # for octocov to parse test execution time
needs:
- test
steps:
- uses: actions/checkout@v4
- uses: actions/download-artifact@v5
with:
pattern: octocov-*
- uses: k1LoW/octocov-action@v1# .octocov.yml
codeToTestRatio:
code:
- "**/*.go"
- "!**/*_test.go"
test:
- "**/*_test.go"
coverage:
paths:
# Name of artifacts uploaded by the test-running jobs.
# lcov.info (or equivalent) will be downloaded in the directory named after the below.
- "octocov-1"
- "octocov-2"
- "octocov-3"
comment:
if: is_pull_request
hideFooterLink: true
deletePrevious: true
summary:
if: true
report:
datastores:
- artifact://${GITHUB_REPOSITORY}
diff:
datastores:
- artifact://${GITHUB_REPOSITORY}- GitHub repository
- GitHub Actions Artifacts
- Amazon S3
- Google Cloud Storage (GCS)
- BigQuery
- Local
octocov ls-files command can be used to list files logged in code coverage report.
octocov view (alias: octocov cat) command can be used to view the file coverage report.
The name of the repository.
It should be in the format owner/repo.
By default, the value of the environment variable GITHUB_REPOSITORY is set.
In case of monorepo, code metrics can be reported to datastore separately by specifying owner/repo/project-a or owner/repo@project-a.
repository: k1LoW/octocovTimeout for octocov execution. (default: 30sec)
timeout: 5minConfiguration for code coverage.
coverage.path: has been deprecated. Please use coverage.paths: instead.
The path to the coverage report file.
If no path is specified, the default path for each coverage format will be scanned.
coverage:
paths:
- tests/coverage.xmlExclude files from the coverage report. Patterns are matched using doublestar glob syntax.
octocov normalizes coverage file paths to git-root-relative paths before matching, so you can write patterns based on the repository directory structure regardless of coverage format.
coverage:
exclude:
- "internal/database/db/*.go"
- "proto/**/*.pb.ts"For backward compatibility, patterns are also matched against the original paths produced by the coverage tool (e.g., Go module paths like github.com/owner/repo/pkg/*.go).
acceptable coverage condition.
coverage:
acceptable: 60%coverage:
acceptable: current >= 60% && diff >= 0.5%The variables that can be used are as follows.
| value | description |
|---|---|
current |
Current code metrics value |
prev |
Previous value. This value is taken from diff.datastores:. |
diff |
The result of current - prev |
patch |
Coverage of the lines changed in the current pull request ("patch coverage") |
coverage:
acceptable: patch >= 80%patch is measured over the changed lines that the coverage report instruments, so changed lines
that no coverage format instruments (blank lines, comments, declarations, ...) are not counted as
uncovered.
patch is measured per changed line, while current and prev are measured in whatever unit the
coverage format instruments. These coincide for line-based formats (LCOV, SimpleCov, Clover,
Cobertura, JaCoCo), but not for Go coverage, which is statement-based. A single statement spanning
several changed lines therefore weighs more in patch than it does in current/prev.
patch requires pull request context: octocov fetches the list of changed files and lines for the
current pull request via the GitHub API. If that context is not available (e.g. not running against
a pull request, or the GitHub API is not accessible), or if the pull request changed no lines that
the coverage report instruments, patch is treated as 100% rather than failing the build.
Only the patch part of the condition is relaxed this way. If you combine patch with
current/prev/diff in one expression (e.g. current >= 80% && patch >= 70%), the
current >= 80% part keeps being enforced even when patch coverage cannot be measured.
Files that the GitHub API does not return a diff for are excluded from patch coverage, because the
changed lines of those files cannot be determined. This happens for binary files, for files whose
diff is too large for the API to include, and for files beyond the limit of the API response
(up to 3000 files for a pull request, and
up to 300 files when comparing against the
default branch outside of a pull request). Such files lower neither the covered lines nor the total
lines of patch.
The file coverage table that octocov reports for a pull request (in the pull request comment, the
job summary, and the pull request body) includes a Patch Coverage column, and its heading shows
the overall patch coverage. This needs no configuration and is independent of
coverage.acceptable:, so the column appears even when the condition does not reference patch.
When the pull request changed no line that the coverage report instruments, the column is dropped
instead of showing - for every file.
octocov diff [REPORT_A] [REPORT_B] --patch also prints the file coverage table of the comparison,
including its patch coverage column, fetching the changed files and lines via the GitHub API. Since
--patch asks for that data explicitly, it fails when the data cannot be fetched. A pull request
that touches no instrumented file is not a failure, so that case reports why the table is missing
and still exits successfully. Without --patch, octocov diff makes no GitHub API calls, as
before.
It is also possible to omit the expression as follows
| Omitted expression | Expanded expression |
|---|---|
60% |
current >= 60% |
> 60% |
current > 60% |
Set this if want to generate the badge self.
The path to the badge.
coverage:
badge:
path: docs/coverage.svgThe text and the background color of the left side of the badge. default: coverage and #24292E.
A color is a hex RGB notation ( #97CA00 ) or one of the shields.io color names ( brightgreen green yellowgreen yellow orange red blue grey lightgrey ).
coverage:
badge:
path: docs/coverage.svg
label: cov
labelColor: "#24292E"The icon image of the badge. A path relative to the config file, or a base64 encoded data: URI. none renders the badge without an icon. default: the octocov icon.
coverage:
badge:
path: docs/coverage.svg
icon: .github/badge-icon.svgThe color of the right side of the badge, as the conditions it is picked by. The color of the first met condition is used, and an entry without if: is met by anything.
The condition is written the same way as coverage.acceptable:, with the measured coverage as current.
coverage:
badge:
path: docs/coverage.svg
colors:
- if: current >= 80%
color: green
- if: 60% # `current >= 60%`
color: yellowgreen
- color: redWhen colors: is not set, or when no condition is met, the built-in thresholds are used ( >= 80% green, >= 60% yellowgreen, >= 40% yellow, >= 20% orange, otherwise red ).
Conditions for measuring code coverage.
coverage:
if: is_default_branchConfiguration for code to test ratio.
Files to count.
codeToTestRatio:
code: # files to count as "Code"
- "**/*.go"
- "!**/*_test.go"
test: # files to count as "Test"
- "**/*_test.go"acceptable ratio condition.
codeToTestRatio:
acceptable: 1:1.2codeToTestRatio:
acceptable: current >= 1.2 && diff >= 0.0The variables that can be used are as follows.
| value | description |
|---|---|
current |
Current code metrics value |
prev |
Previous value. This value is taken from diff.datastores:. |
diff |
The result of current - prev |
It is also possible to omit the expression as follows
| Omitted expression | Expanded expression |
|---|---|
1:1.2 |
current >= 1.2 |
> 1:1.2 |
current > 1.2 |
Set this if want to generate the badge self.
The path to the badge.
codeToTestRatio:
badge:
path: docs/ratio.svgcodeToTestRatio.badge.label: codeToTestRatio.badge.labelColor: codeToTestRatio.badge.icon: codeToTestRatio.badge.colors:
The appearance of the badge, written the same way as coverage.badge:. The condition of colors: takes the measured ratio as current, written the same way as codeToTestRatio.acceptable:.
codeToTestRatio:
badge:
path: docs/ratio.svg
colors:
- if: 1:1.2
color: green
- color: redThe built-in thresholds are used when colors: is not set or when no condition is met ( >= 1.2 green, >= 1.0 yellowgreen, >= 0.8 yellow, >= 0.6 orange, otherwise red ).
Conditions for measuring code to test ratio.
codeToTestRatio:
if: is_default_branchConfiguration for test execution time.
acceptable time condition.
testExecutionTime:
acceptable: 1mintestExecutionTime:
acceptable: current <= 1min && diff <= 1secThe variables that can be used are as follows.
| value | description |
|---|---|
current |
Current code metrics value |
prev |
Previous value. This value is taken from diff.datastores:. |
diff |
The result of current - prev |
It is also possible to omit the expression as follows
| Omitted expression | Expanded expression |
|---|---|
1min |
current <= 1min |
< 1min |
current < 1min |
The name of the step to measure the execution time.
testExecutionTime:
steps:
- Run test
- Run slow testSteps with a matching name are collected from every job in the workflow run, and their execution times are merged (overlapping time ranges are counted only once). This makes it possible to measure the total test execution time when the tests are split across multiple jobs, for example by a GitHub Actions matrix. See Merging reports from multiple GitHub Action jobs.
If not specified, the step where the coverage report file is generated is used as the measurement target.
Set this if want to generate the badge self.
The path to the badge.
testExecutionTime:
badge:
path: docs/time.svgtestExecutionTime.badge.label: testExecutionTime.badge.labelColor: testExecutionTime.badge.icon: testExecutionTime.badge.colors:
The appearance of the badge, written the same way as coverage.badge:. The condition of colors: takes the measured time as current, written the same way as testExecutionTime.acceptable:.
testExecutionTime:
badge:
path: docs/time.svg
colors:
- if: 1min
color: green
- color: redThe built-in thresholds are used when colors: is not set or when no condition is met ( < 5min green, < 10min yellowgreen, < 15min yellow, < 20min orange, otherwise red ).
Conditions for measuring test execution time.
testExecutionTime:
if: is_pull_requestConfiguration for git push files self.
Conditions for pushing files.
# .octocov.yml
push:
if: is_default_branchThe variables available in the if section are here.
message for commit.
# .octocov.yml
push:
message: Update by octocov [skip ci]Set this if want to comment report to pull request
Hide footer octocov link.
comment:
hideFooterLink: trueHide the octocov.dev links the coverage values carry.
comment:
hideCoverageLink: trueExpand the collapsible sections of the report ( the Details of the comparison with the previous report, and the Metadata of each custom metric set ) instead of rendering them folded.
The Details section is rendered only when a previous report was found to compare against. The file coverage table folds itself once the pull request touches enough files, which this does not change, since that fold is what keeps the comment inside what GitHub will render.
comment:
expandDetails: trueDelete previous code metrics report comments instead of hiding them
comment:
deletePrevious: trueUpdate the previous code metrics report comment instead of creating a new one.
comment:
updatePrevious: trueThis is useful when you want octocov to keep a single managed pull request comment up to date.
Add message to code metrics report comments.
comment:
message: See [coverage html](https://github.com/k1LoW/octocov).Conditions for commenting report.
# .octocov.yml
comment:
if: is_pull_requestThe variables available in the if section are here.
Set this if want to add report to job summary page.
Hide footer octocov link.
summary:
hideFooterLink: trueHide the octocov.dev links the coverage values carry.
summary:
hideCoverageLink: trueExpand the collapsible sections of the report ( the Details of the comparison with the previous report, and the Metadata of each custom metric set ) instead of rendering them folded.
summary:
expandDetails: trueAdd message to report.
summary:
message: See [coverage html](https://github.com/k1LoW/octocov).Conditions for adding report to job summary page.
# .octocov.yml
summary:
if: trueThe variables available in the if section are here.
Set this if want to insert report to body of pull request.
Hide footer octocov link.
body:
hideFooterLink: trueHide the octocov.dev links the coverage values carry.
body:
hideCoverageLink: trueExpand the collapsible sections of the report ( the Details of the comparison with the previous report, and the Metadata of each custom metric set ) instead of rendering them folded.
body:
expandDetails: trueAdd message to report.
body:
message: See [coverage html](https://github.com/k1LoW/octocov).Conditions for inserting report body of pull request.
# .octocov.yml
body:
if: is_pull_requestThe variables available in the if section are here.
Configuration for comparing reports.
Path of the report to compare.
diff:
path: path/to/coverage.ymldiff:
path: path/to/report.jsonDatastores where the report to be compared is stored.
diff:
datastores:
- local://.octocov # Use .octocov/owner/repo/report.json
- s3://my-bucket/reports # Use s3://my-bucket/reports/owner/repo/report.jsonConditions for comparing reports
# .octocov.yml
diff:
if: is_pull_request
path: path/to/report.jsonThe variables available in the if section are here.
Configuration for reporting to datastores.
Path to save the report.
report:
path: path/to/report.jsonDatastores where the reports are stored.
report:
datastores:
- github://owner/coverages/reports
- s3://bucket/reportsUse github:// scheme.
github://[owner]/[repo]@[branch]/[prefix]
Required environment variables:
GITHUB_TOKENorOCTOCOV_GITHUB_TOKENGITHUB_REPOSITORYorOCTOCOV_GITHUB_REPOSITORYGITHUB_API_URLorOCTOCOV_GITHUB_API_URL(optional)
Use artifact:// or artifacts:// scheme.
artifact://[owner]/[repo]/[artifactName]
artifact://[owner]/[repo]/[artifactName]artifact://[owner]/[repo]( default artifactName:octocov-report)
The report of a ref other than the default branch is stored under an artifact of its own, named by appending @ and the ref to the artifact name, with the characters an artifact name may not hold replaced by _ ( e.g. octocov-report@refs_pull_123, octocov-report@refs_heads_feat_x ).
Note that reporting to the artifact can only be sent from the GitHub Actions of the same repository.
Required environment variables:
GITHUB_TOKENorOCTOCOV_GITHUB_TOKENGITHUB_REPOSITORYorOCTOCOV_GITHUB_REPOSITORYGITHUB_API_URLorOCTOCOV_GITHUB_API_URL(optional)
Use s3:// scheme.
s3://[bucket]/[prefix]
Required permission:
s3:PutObject
Required environment variables:
AWS_ACCESS_KEY_IDorOCTOCOV_AWS_ACCESS_KEY_IDAWS_SECRET_ACCESS_KEYorOCTOCOV_AWS_SECRET_ACCESS_KEYAWS_SESSION_TOKENorOCTOCOV_AWS_SESSION_TOKEN(optional)
Use gs:// scheme.
gs://[bucket]/[prefix]
Required permission:
storage.objects.createstorage.objects.delete
Required environment variables:
GOOGLE_APPLICATION_CREDENTIALSorGOOGLE_APPLICATION_CREDENTIALS_JSONorOCTOCOV_GOOGLE_APPLICATION_CREDENTIALSorOCTOCOV_GOOGLE_APPLICATION_CREDENTIALS_JSON
Use bq:// scheme.
bq://[project ID]/[dataset ID]/[table]
Required permission:
bigquery.datasets.getbigquery.tables.getbigquery.tables.updateData
Required environment variables:
GOOGLE_APPLICATION_CREDENTIALSorGOOGLE_APPLICATION_CREDENTIALS_JSONorOCTOCOV_GOOGLE_APPLICATION_CREDENTIALSorOCTOCOV_GOOGLE_APPLICATION_CREDENTIALS_JSON
Datastore schema:
If you want to create a table, execute the following command ( require bigquery.datasets.create ).
$ octocov migrate-bq-tableNote: Only works with
report.datastoresorcentral.reReport.datastores
Use mackerel:// or mkr:// scheme.
mackerel://[Service Name]
Required permission:
readwrite
Required environment variables:
MACKEREL_API_KEYorOCTOCOV_MACKEREL_API_KEY
Use local:// or file:// scheme.
local://[path]
Example:
If the absolute path of .octocov.yml is /path/to/.octocov.yml
local://reports.../path/to/reportsdirectorylocal://./reports.../path/to/reportsdirectorylocal://../reports.../path/reportsdirectorylocal:///reports.../reportsdirectory.
Conditions for storing a report.
# .octocov.yml
report:
if: env.GITHUB_REF == 'refs/heads/main'
datastores:
- github://owner/coverages/reportsThe variables available in the if section are here.
Note: It supports expr-lang/expr expressions.
The variables available in the if section are as follows
| Variable name | Type | Description |
|---|---|---|
year |
int |
Year of current time (UTC) |
month |
int |
Month of current time (UTC) |
day |
int |
Day of current time (UTC) |
hour |
int |
Hour of current time (UTC) |
weekday |
int |
Weekday of current time (UTC) (Sunday = 0, ...) |
github.event_name |
string |
Event name of GitHub Actions ( ex. issues, pull_request ) |
github.event |
object |
Detailed data for each event of GitHub Actions (ex. github.event.action, github.event.label.name ) |
env.<env_name> |
string |
The value of a specific environment variable |
is_pull_request |
boolean |
Whether the job is related to an pull request (ex. a job fired by on.push will be true if it is related to a pull request) |
is_draft |
boolean |
Whether the job is related to a draft pull request |
labels |
array |
Labels that are set for the pull request |
is_default_branch |
boolean |
Whether the job is related to default branch of repository |
Note: When central mode is enabled, other functions are automatically turned off.
The root directory or index file ( index file example ) path of collected coverage reports pages. default: .
central:
root: path/toDatastore paths (URLs) where reports are stored. default: local://reports
central:
reports:
datastores:
- local://reports
- gs://my-gcs-bucket/reportsWhen using GitHub Actions Artifacts as a datastore, perform badge generation via on.schedule.
# .octocov.yml
report:
datastores:
- artifact://${GITHUB_REPOSITORY}# .octocov.yml for central repo
central:
reports:
datastores:
- artifact://owner/repo
- artifact://owner/other-repo
- artifact://owner/another-repo
[...]
push:Code metrics and badges of my open source projects using octocov central mode is here.
When using the central repository as a datastore, perform badge generation via on.push.
# .octocov.yml
report:
datastores:
- github://owner/central-repo/reports# .octocov.yml for central repo
central:
reports:
datastores:
- github://owner/central-repo/reports
push:or
# .octocov.yml for central repo
central:
reports:
datastores:
- local://reports
push:When using the S3 bucket as a datastore, perform badge generation via on.schedule.
# .octocov.yml
report:
datastores:
- s3://my-s3-bucket/reports# .octocov.yml for central repo
central:
reports:
datastores:
- s3://my-s3-bucket/reports
push:Required permission (Central Repo):
s3:GetObjects3:ListObject
Required environment variables (Central Repo):
AWS_ACCESS_KEY_IDorOCTOCOV_AWS_ACCESS_KEY_IDAWS_SECRET_ACCESS_KEYorOCTOCOV_AWS_SECRET_ACCESS_KEYAWS_SESSION_TOKENorOCTOCOV_AWS_SESSION_TOKEN(optional)
When using the GCS bucket as a datastore, perform badge generation via on.schedule.
# .octocov.yml
report:
datastores:
- gs://my-gcs-bucket/reports# .octocov.yml for central repo
central:
reports:
datastores:
- gs://my-gcs-bucket/reports
push:Required permission (Central Repo):
storage.objects.getstorage.objects.liststorage.buckets.get
Required environment variables (Central Repo):
GOOGLE_APPLICATION_CREDENTIALSorGOOGLE_APPLICATION_CREDENTIALS_JSONorOCTOCOV_GOOGLE_APPLICATION_CREDENTIALSorOCTOCOV_GOOGLE_APPLICATION_CREDENTIALS_JSON
When using the BigQuery table as a datastore, perform badge generation via on.schedule.
# .octocov.yml
report:
datastores:
- bq://my-project/my-dataset/reports# .octocov.yml for central repo
central:
reports:
datastores:
- bq://my-project/my-dataset/reports
push:Required permission (Central Repo):
bigquery.jobs.createbigquery.tables.getData
Required environment variables (Central Repo):
GOOGLE_APPLICATION_CREDENTIALSorGOOGLE_APPLICATION_CREDENTIALS_JSONorOCTOCOV_GOOGLE_APPLICATION_CREDENTIALSorOCTOCOV_GOOGLE_APPLICATION_CREDENTIALS_JSON
Datastore paths (URLs) where badges are generated. default: local://badges
central:
badges:
datastores:
- local://badges
- s3://my-s3-buckets/badgesThe appearance of the badges generated in central mode, written the same way as coverage.badge: except for path:, which central mode decides.
A report does not carry the configuration of the repository it was collected from, so the badges of every repository are generated as configured here.
central:
badges:
datastores:
- local://badges
coverage:
icon: none
colors:
- if: 80%
color: green
- color: redConfiguration for git push index file and badges self.
Conditions for central mode.
# .octocov.yml
central:
if: env.GITHUB_REF == 'refs/heads/main'
reports:
datastores:
- s3://my-s3-bucket/reportsThe variables available in the if section are here.
Store collected reports in yet another datastores.
Conditions for re storing reports.
Datastores where the reports are re-stored.
octocov supports multiple coverage report formats.
And octocov searches for the default path for each format.
If you want to specify the path of the report file, set coverage.path
coverage:
paths:
- /path/to/coverage.txtDefault path: coverage.out
Default path: coverage/lcov.info
Support SF DA only
Default path: coverage/.resultset.json
Default path: coverage.xml
Default path: coverage.xml
Default path: build/reports/jacoco/test/jacocoTestReport.xml
- Code Coverage
- Code to Test Ratio
- Test Execution Time (on GitHub Actions only)
octocov accepts custom metrics in addition to the three supporting metrics.
Specify the path to the custom metrics JSON file in an environment variable prefixed with OCTOCOV_CUSTOM_METRICS_ to collect the code metrics at the same time.
The JSON schema for custom metrics can be found here.
If there are multiple custom metrics JSON files, specify each file path in a separate environment variable (example here) or combine the JSONs that satisfy the JSON schema into an array.
You can set acceptable conditions for custom metrics using the acceptables array within your JSON file.
JSON file format with acceptables:
{
"key": "performance_metrics",
"name": "Performance Metrics",
"metrics": [
{
"key": "response_time",
"name": "Response Time",
"value": 250.0,
"unit": "ms"
},
{
"key": "throughput",
"name": "Throughput",
"value": 1500.0,
"unit": "req/s"
},
{
"key": "error_rate",
"name": "Error Rate",
"value": 0.5,
"unit": "%"
}
],
"acceptables": [
"current.response_time < 300",
"current.throughput > 1000",
"current.error_rate < 1.0",
"current.response_time <= prev.response_time"
]
}Available variables in acceptables conditions:
| Variable | Description |
|---|---|
current.{metric_key} |
Current metric value for the specified key |
prev.{metric_key} |
Previous metric value. This value is taken from diff.datastores: |
diff.{metric_key} |
The result of current.{metric_key} - prev.{metric_key} |
If conditions are not met:
$ octocov
Error: custom metrics "performance_metrics" not acceptable condition (`current.response_time < 300`)octocov detect pull request number following order.
- Get pull request number from
GITHUB_PULL_REQUEST_NUMBERorOCTOCOV_GITHUB_PULL_REQUEST_NUMBER. - Get pull request number from
GITHUB_REF( e.g.refs/pull/1/merge). - Get branch name from
GITHUB_REF( e.g.refs/heads/branch/branch/name) and detect pull request number using GitHub API.
If an environment variable with prefix OCTOCOV_ is set, it is used as an unprefixed environment variable in octocov.
For example, if OCTOCOV_GITHUB_REF is set, it is handled as GITHUB_REF in octocov.
This feature allows environment variables that cannot normally be overridden to be changed on octocov.
deb:
$ export OCTOCOV_VERSION=X.X.X
$ curl -o octocov.deb -L https://github.com/k1LoW/octocov/releases/download/v$OCTOCOV_VERSION/octocov_$OCTOCOV_VERSION-1_amd64.deb
$ dpkg -i octocov.debRPM:
$ export OCTOCOV_VERSION=X.X.X
$ yum install https://github.com/k1LoW/octocov/releases/download/v$OCTOCOV_VERSION/octocov_$OCTOCOV_VERSION-1_amd64.rpmapk:
$ export OCTOCOV_VERSION=X.X.X
$ curl -o octocov.apk -L https://github.com/k1LoW/octocov/releases/download/v$OCTOCOV_VERSION/octocov_$OCTOCOV_VERSION-1_amd64.apk
$ apk add octocov.apkhomebrew tap:
$ brew install k1LoW/tap/octocovaqua:
$ aqua g -i k1LoW/octocovmanually:
Download binary from releases page
go install:
$ go install github.com/k1LoW/octocov@latestdocker:
$ docker pull ghcr.io/k1low/octocov:latest
