rust/lintcheck
bors be8234098b Auto merge of #13139 - xFrednet:lintcheck-limit-summery-output, r=Alexendoo
Lintcheck: Rework and limit diff output for GH's CI

### Background

While working on https://github.com/rust-lang/rust-clippy/pull/13136 I found an amazing limitation of GH's CI. The summary can at most have be 1MB of text. Here is the warning message:

> $GITHUB_STEP_SUMMARY upload aborted, supports content up to a size of 1024k, got 46731k. For more information see: https://docs.github.com/actions/using-workflows/workflow-commands-for-github-actions#adding-a-markdown-summary

[The PR](https://github.com/rust-lang/rust-clippy/pull/13136) produced a *casual* 61808 changes. Guess that's why those lints are not *warn-by-default* :P.

### Changes:

This PR limits the lintcheck diff output in two ways.

1. The diff is limited to 200 messages per lint per section. Hidden messages are indicated by a message at the end of the section.
2. The output is first written to a file and only the first 1MB is written to ` >> $GITHUB_STEP_SUMMARY`. The entire file is also written to the normal CI log. This helps for cases where several lints change and the total size exceeds the 1MB limit.

An example of these changes can be seen here: https://github.com/xFrednet/rust-clippy/actions/runs/10028799118?pr=4

---

changelog: none

r? `@Alexendoo`

Sorry for bombarding you with so many PR's lately 😅 Feel free to pass some of you reviews to me.
2024-07-25 12:11:35 +00:00
..
src Auto merge of #13139 - xFrednet:lintcheck-limit-summery-output, r=Alexendoo 2024-07-25 12:11:35 +00:00
Cargo.toml Lintcheck: Review comments <3 2024-07-24 22:45:41 +02:00
ci_crates.toml Lintcheck: New testset for Clippy's CI (200 crates for ~10 min) 2024-07-19 19:52:07 +02:00
lintcheck_crates.toml Lintcheck: Update the default testset for local lintcheck 2024-07-19 19:52:03 +02:00
README.md Lintcheck: Update lintcheck documentation 2024-07-07 18:42:09 +02:00
test_sources.toml Merge commit '6ed6f1e6a1a8f414ba7e6d9b8222e7e5a1686e42' into clippyup 2021-03-12 15:30:50 +01:00

cargo lintcheck

Runs Clippy on a fixed set of crates read from lintcheck/lintcheck_crates.toml and saves logs of the lint warnings into the repo. We can then check the diff and spot new or disappearing warnings.

From the repo root, run:

cargo lintcheck

or

cargo run --target-dir lintcheck/target --manifest-path lintcheck/Cargo.toml

By default, the logs will be saved into lintcheck-logs/lintcheck_crates_logs.txt.

You can set a custom sources.toml by adding --crates-toml custom.toml or using LINTCHECK_TOML="custom.toml" where custom.toml must be a relative path from the repo root.

The results will then be saved to lintcheck-logs/custom_logs.toml.

The custom.toml file may be built using https://crates.io recently most downloaded crates by using cargo lintcheck popular. For example, to retrieve the 200 recently most downloaded crates:

cargo lintcheck popular -n 200 custom.toml

Note: Lintcheck isn't sandboxed. Only use it to check crates that you trust or sandbox it manually.

Configuring the Crate Sources

The sources to check are saved in a toml file. There are three types of sources.

  1. Crates-io Source

    bitflags = {name = "bitflags", versions = ['1.2.1']}
    

    Requires a "name" and one or multiple "versions" to be checked.

  2. git Source

    puffin = {name = "puffin", git_url = "https://github.com/EmbarkStudios/puffin", git_hash = "02dd4a3"}
    

    Requires a name, the url to the repo and unique identifier of a commit, branch or tag which is checked out before linting. There is no way to always check HEAD because that would lead to changing lint-results as the repo would get updated. If git_url or git_hash is missing, an error will be thrown.

  3. Local Dependency

    clippy = {name = "clippy", path = "/home/user/clippy"}
    

    For when you want to add a repository that is not published yet.

Command Line Options (optional)

clap = {name = "clap", versions = ['4.5.8'], options = ['-Fderive']}

It is possible to specify command line options for each crate. This makes it possible to enable or disable features.

Fix mode

You can run cargo lintcheck --fix which will run Clippy with --fix and print a warning if Clippy's suggestions fail to apply (if the resulting code does not build). This lets us spot bad suggestions or false positives automatically in some cases.

Note: Fix mode implies --all-targets, so it can fix as much code as it can.

Please note that the target dir should be cleaned afterwards since Clippy will modify the downloaded sources which can lead to unexpected results when running lintcheck again afterwards.

Recursive mode

You can run cargo lintcheck --recursive to also run Clippy on the dependencies of the crates listed in the crates source .toml. e.g. adding rand 0.8.5 would also lint rand_core, rand_chacha, etc.

Particularly slow crates in the dependency graph can be ignored using recursive.ignore:

[crates]
cargo = {name = "cargo", versions = ['0.64.0']}

[recursive]
ignore = [
    "unicode-normalization",
]