feat(lint): libclang-backed token layer

Adds LintContext::Tokens() and friends, backed by clang_tokenize, as the
substrate the rules will move onto. Nothing consumes it yet.

libclang is dlopen'd rather than linked: -lclang would break the mingw and
MSVC cross-builds at link time and would put a libclang.so.NN runtime
dependency into the otherwise self-contained release tarballs. The clang-c
header is used for its declarations only, and the function-pointer table is
typed with decltype so the signatures cannot drift from the real API.

Three properties this buys that the hand-rolled scanners could not have:

  - a raw string literal or block comment is ONE token, so the documented
    "raw string literals are not recognized" limitation goes away;
  - `//` inside a literal is not a comment, so LineHasComment() replaces the
    Line(n).contains("//") probes that false-positive on it;
  - tokens cover preprocessor branches that are inactive for the host, since
    clang_tokenize lexes rather than evaluates #if. Token rules therefore
    keep seeing every platform's code, which an AST could not offer.

The parse backing the tokenizer is expected to fail on module units — no
PCMs, no build flags — and that is fine, because lexing has no semantic
prerequisites. Verified in the new tests.

LintSummary::Clean() now counts `errors`. It previously ignored them, so an
infrastructure failure that produced no findings reported clean and exited
0; a missing libclang would have been exactly that.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
This commit is contained in:
Jorijn van der Graaf 2026-07-27 02:54:38 +02:00
commit 5888af29ed
4 changed files with 387 additions and 2 deletions

View file

@ -39,10 +39,13 @@ export namespace Crafter {
std::vector<std::filesystem::path> changedFiles;
std::size_t filesLinted = 0;
std::size_t rulesRun = 0; // rules remaining after glob filter
std::size_t errors = 0; // rule exceptions + write failures
// Rule exceptions, write failures, and infrastructure failures such as
// libclang not loading. Counted in Clean() so a run that could not do
// its job never looks like a run that found nothing.
std::size_t errors = 0;
bool noRulesDefined = false; // project registered no rules at all
// Host-side only (like TestSummary::AllPassed), safe as in-class inline.
bool Clean() const { return findings.empty() && !noRulesDefined; }
bool Clean() const { return findings.empty() && !noRulesDefined && errors == 0; }
};
// Run the project's lint rules over its own sources: module interfaces