Catcrafts/fp6-img
Watch
2
0
Fork
You've already forked fp6-img
 0
Code Issues Pull requests Projects Releases 1 Packages Wiki Activity Actions

temp/libcamera: r9 - zone-gated AWB for the simple IPA (patch 0016)
Some checks failed
image / image (push) Failing after 3h4m27s
Details

Browse source 
Divide the softISP statistics window into 16x12 zones and, when a white
point locus is tuned, vote only with zones whose implied gains lie within
zoneGateMargin of the locus - a large coloured light source can no longer
drag the white balance. Validated on-phone 2026-08-24: keyboard-scene A/B
shows a ~1700 K estimate shift caused solely by the gate (desk renders
neutral instead of blue); torch-toggle stability equal or better than r8
(settle 1.35-1.79 s, ON-edge repeatability +-0.011, CCM churn 21 vs 30
switches); AF regression clean. Evidence:
journal/camera/captures/2026-08-24-awb-zonegate-validation.md

Assisted-by: Claude:claude-fable-5
This commit is contained in:
Jorijn van der Graaf 2026-08-24 01:05:14 +02:00
commit d91963975a
2 changed files with 368 additions and 1 deletions
Download patch file Download diff file Expand all files Collapse all files

365
aports/temp/libcamera/0016-ipa-simple-awb-Gate-the-gray-world-estimate-on-near-.patch Normal file
View file

@ -0,0 +1,365 @@
From 3e109c99ec651719311580f574d8a152a5a09680 Mon Sep 17 00:00:00 2001
From: Jorijn van der Graaf <jorijnvdgraaf@catcrafts.net>
Date: Sun, 23 Aug 2026 22:32:48 +0200
Subject: [PATCH] ipa: simple: awb: Gate the gray-world estimate on
near-neutral zones
A single large coloured area - typically a coloured light source, e.g.
an RGB keyboard backlight filling the bottom of the frame - defeats
whole-frame gray world by design: the frame average it produces is
statistically identical to that of a neutral scene under some other
illuminant, so the locus-clamped estimate settles confidently at the
wrong end of the locus (measured 2026-08-17: a cyan-backlit keyboard
renders the ambient-lit desk red; stable, uniform and wrong).
Divide the statistics window into 16x12 zones and collect per-zone RGB
sums alongside the existing whole-frame sums, at the same sampling
density. In the Awb algorithm, when a white point locus is tuned, vote
only with the zones whose implied gains lie within zoneGateMargin of
the locus - zones some plausible illuminant could render gray - and
fall back to the whole-frame sums when fewer than four zones qualify.
Zones that are too dark, clipped, or carry too few samples are
excluded from the vote.
The gate runs ahead of the existing locus clamp and damping, which are
unchanged. Without whitePoints tuning the zone statistics are still
collected but the algorithm behaves exactly as before.
Assisted-by: Claude:claude-fable-5
Signed-off-by: Jorijn van der Graaf <jorijnvdgraaf@catcrafts.net>
---
.../internal/software_isp/swisp_stats.h | 28 +++++
src/ipa/simple/algorithms/awb.cpp | 119 +++++++++++++++++-
src/ipa/simple/algorithms/awb.h | 4 +
src/libcamera/software_isp/swstats_cpu.cpp | 47 ++++++-
4 files changed, 193 insertions(+), 5 deletions(-)
diff --git a/include/libcamera/internal/software_isp/swisp_stats.h b/include/libcamera/internal/software_isp/swisp_stats.h
index c2ec20e..22cdd0e 100644
--- a/include/libcamera/internal/software_isp/swisp_stats.h
+++ b/include/libcamera/internal/software_isp/swisp_stats.h
@@ -47,6 +47,34 @@ struct SwIspStats {
* \brief Holds the sharpness of an image
*/
uint64_t sharpness;
+ /**
+ * \brief Number of AWB statistics zone columns
+ */
+ static constexpr unsigned int kAwbZoneCols = 16;
+ /**
+ * \brief Number of AWB statistics zone rows
+ */
+ static constexpr unsigned int kAwbZoneRows = 12;
+ /**
+ * \brief Per-zone colour channel sums and sample count
+ *
+ * The sums are in the same scale as sum_. uint32_t is large enough:
+ * a zone of a 12-bit 8192x6144 frame at the stats sampling density
+ * sums to at most ~2^26.
+ */
+ struct AwbZone {
+ uint32_t sumR;
+ uint32_t sumG;
+ uint32_t sumB;
+ uint32_t count;
+ };
+ /**
+ * \brief Colour channel sums of the sampled pixels per image zone
+ *
+ * The statistics window is divided into kAwbZoneCols x kAwbZoneRows
+ * equally sized zones, stored in row-major order.
+ */
+ std::array<AwbZone, kAwbZoneCols * kAwbZoneRows> awbZones;
};
} /* namespace libcamera */
diff --git a/src/ipa/simple/algorithms/awb.cpp b/src/ipa/simple/algorithms/awb.cpp
index 83dcece..e47db76 100644
--- a/src/ipa/simple/algorithms/awb.cpp
+++ b/src/ipa/simple/algorithms/awb.cpp
@@ -43,6 +43,22 @@ constexpr unsigned int kFastConvergeFrames = 3;
/* Start from the white point closest to daylight. */
constexpr double kInitialCT = 5000.0;
+/*
+ * Zone gating thresholds. A zone votes only when it carries enough samples
+ * to trust its means (kZoneMinSamples), no channel mean is close to clipping
+ * (kZoneClipMean, in the 8-bit scale of the statistics), and the green mean
+ * clears the black level by at least kZoneMinSignal. The implied per-zone
+ * gains are capped at kZoneMaxGain to keep the locus distance finite for
+ * almost-black channels. Fewer than kZoneMinIncluded voting zones means the
+ * scene carries no usable spatial signal and the whole-frame estimate is
+ * used instead.
+ */
+constexpr unsigned int kZoneMinSamples = 16;
+constexpr double kZoneClipMean = 250.0;
+constexpr double kZoneMinSignal = 2.0;
+constexpr double kZoneMaxGain = 8.0;
+constexpr unsigned int kZoneMinIncluded = 4;
+
} /* namespace */
namespace libcamera {
@@ -70,6 +86,18 @@ namespace ipa::soft::algorithms {
* temperature fed to the Ccm algorithm is interpolated from the applied
* gains' position along the locus. Without whitePoints the plain
* gray-world algorithm runs unchanged.
+ *
+ * With whitePoints two further keys enable spatial (zone-gated) AWB:
+ *
+ * zoneGating: true # vote with near-neutral zones only
+ * zoneGateMargin: 0.35 # max locus distance for a zone to vote
+ *
+ * The gray-world sums are then taken from the statistics zones whose
+ * implied gains lie within zoneGateMargin (default clampMargin) of the
+ * locus, i.e. zones some plausible illuminant could render gray, so a
+ * large coloured area - typically a coloured light source - no longer
+ * drags the white balance. When too few zones qualify the whole-frame
+ * sums are used as before.
*/
int Awb::init([[maybe_unused]] IPAContext &context, const ValueNode &tuningData)
{
@@ -120,11 +148,21 @@ int Awb::init([[maybe_unused]] IPAContext &context, const ValueNode &tuningData)
return -EINVAL;
}
+ zoneGating_ = tuningData["zoneGating"].get<bool>(true);
+ zoneGateMargin_ = tuningData["zoneGateMargin"].get<double>(clampMargin_);
+ if (zoneGateMargin_ <= 0.0) {
+ LOG(IPASoftAwb, Error) << "zoneGateMargin must be positive";
+ whitePoints_.clear();
+ return -EINVAL;
+ }
+
LOG(IPASoftAwb, Info)
<< "White point locus: " << whitePoints_.size() << " points, "
<< whitePoints_.front().ct << "-" << whitePoints_.back().ct
<< " K, clamp margin " << clampMargin_
- << ", damping " << damping_;
+ << ", damping " << damping_
+ << ", zone gating " << (zoneGating_ ? "on" : "off")
+ << " margin " << zoneGateMargin_;
return 0;
}
@@ -202,6 +240,73 @@ Awb::LocusPosition Awb::closestOnLocus(double rGain, double bGain) const
return best;
}
+/*
+ * Compute the gray-world sums from the near-neutral image zones only.
+ *
+ * A large coloured area - typically a coloured light source such as an RGB
+ * keyboard backlight - defeats whole-frame averaging by design: the frame
+ * average it produces is statistically identical to that of a neutral scene
+ * under some other illuminant, so the estimate is confidently wrong. Per
+ * zone the distinction is visible: a zone votes only when the gains that
+ * would render it gray lie close to the calibrated white point locus, i.e.
+ * when some plausible illuminant could have produced it from a neutral
+ * surface. Zones that are too dark, clipped or strongly coloured are
+ * excluded. Returns false when too few zones vote; the caller then falls
+ * back to the whole-frame estimate.
+ */
+bool Awb::gatedEstimate(const SwIspStats &stats, double blackLevel,
+ double &rTarget, double &bTarget) const
+{
+ double sumR = 0.0, sumG = 0.0, sumB = 0.0;
+ unsigned int eligible = 0, included = 0;
+
+ for (const SwIspStats::AwbZone &zone : stats.awbZones) {
+ if (zone.count < kZoneMinSamples)
+ continue;
+
+ const double meanR = static_cast<double>(zone.sumR) / zone.count;
+ const double meanG = static_cast<double>(zone.sumG) / zone.count;
+ const double meanB = static_cast<double>(zone.sumB) / zone.count;
+
+ /* A clipped channel no longer measures colour. */
+ if (meanR > kZoneClipMean || meanG > kZoneClipMean ||
+ meanB > kZoneClipMean)
+ continue;
+
+ const double r = meanR - blackLevel;
+ const double g = meanG - blackLevel;
+ const double b = meanB - blackLevel;
+ if (g < kZoneMinSignal)
+ continue;
+
+ eligible++;
+
+ const double gR = g / std::max(r, g / kZoneMaxGain);
+ const double gB = g / std::max(b, g / kZoneMaxGain);
+ if (closestOnLocus(gR, gB).distance > zoneGateMargin_)
+ continue;
+
+ included++;
+ sumR += r * zone.count;
+ sumG += g * zone.count;
+ sumB += b * zone.count;
+ }
+
+ if (included < kZoneMinIncluded || sumR <= 0.0 || sumB <= 0.0) {
+ LOG(IPASoftAwb, Debug)
+ << "zone gate: " << included << "/" << eligible
+ << " zones, falling back to whole-frame";
+ return false;
+ }
+
+ LOG(IPASoftAwb, Debug)
+ << "zone gate: " << included << "/" << eligible << " zones vote";
+
+ rTarget = sumG / sumR;
+ bTarget = sumG / sumB;
+ return true;
+}
+
void Awb::process(IPAContext &context,
[[maybe_unused]] const uint32_t frame,
IPAFrameContext &frameContext,
@@ -258,11 +363,17 @@ void Awb::process(IPAContext &context,
}
/*
- * Gray-world estimate. The sums are at least minValid, so the
+ * Gray-world estimate, from the near-neutral zones when enough of
+ * them pass the chromaticity gate, from the whole-frame sums
+ * otherwise. The whole-frame sums are at least minValid, so the
* divisions are safe; the locus clamp below bounds the result.
*/
- double rTarget = static_cast<double>(sum.g()) / sum.r();
- double bTarget = static_cast<double>(sum.g()) / sum.b();
+ double rTarget, bTarget;
+ if (!zoneGating_ ||
+ !gatedEstimate(*stats, blackLevel, rTarget, bTarget)) {
+ rTarget = static_cast<double>(sum.g()) / sum.r();
+ bTarget = static_cast<double>(sum.g()) / sum.b();
+ }
/* Clamp the estimate to within clampMargin_ of the locus. */
const LocusPosition projection = closestOnLocus(rTarget, bTarget);
diff --git a/src/ipa/simple/algorithms/awb.h b/src/ipa/simple/algorithms/awb.h
index ef4e4f3..f86b31f 100644
--- a/src/ipa/simple/algorithms/awb.h
+++ b/src/ipa/simple/algorithms/awb.h
@@ -49,6 +49,8 @@ private:
};
LocusPosition closestOnLocus(double rGain, double bGain) const;
+ bool gatedEstimate(const SwIspStats &stats, double blackLevel,
+ double &rTarget, double &bTarget) const;
/*
* Calibrated white point locus in (R gain, B gain) space, ordered by
@@ -58,6 +60,8 @@ private:
std::vector<WhitePoint> whitePoints_;
double clampMargin_ = 0.0;
double damping_ = 0.0;
+ bool zoneGating_ = false;
+ double zoneGateMargin_ = 0.0;
unsigned int fastFrames_ = 0;
};
diff --git a/src/libcamera/software_isp/swstats_cpu.cpp b/src/libcamera/software_isp/swstats_cpu.cpp
index 2cbdbef..ed32c47 100644
--- a/src/libcamera/software_isp/swstats_cpu.cpp
+++ b/src/libcamera/software_isp/swstats_cpu.cpp
@@ -11,6 +11,8 @@
#include "libcamera/internal/software_isp/swstats_cpu.h"
+#include <algorithm>
+
#include <libcamera/base/log.h>
#include <libcamera/stream.h>
@@ -192,7 +194,19 @@ static constexpr unsigned int kBlueYMul = 29; /* 0.114 * 256 */
const unsigned int sharpXBegin = (lineLength) * 2 / 5; \
const unsigned int sharpXEnd = (lineLength) * 3 / 5; \
const bool sharpRow = y >= window_.height * 2 / 5 && \
- y < window_.height * 3 / 5;
+ y < window_.height * 3 / 5; \
+ uint32_t zSumR = 0; \
+ uint32_t zSumG = 0; \
+ uint32_t zSumB = 0; \
+ uint32_t zCount = 0; \
+ unsigned int zCol = 0; \
+ const unsigned int zLineLength = (lineLength); \
+ unsigned int zEndX = zLineLength / SwIspStats::kAwbZoneCols; \
+ SwIspStats::AwbZone *zRow = &stats.awbZones[ \
+ std::min(y * SwIspStats::kAwbZoneRows / \
+ static_cast<unsigned int>(window_.height), \
+ SwIspStats::kAwbZoneRows - 1) * \
+ SwIspStats::kAwbZoneCols];
/*
* Sharpness is the sum of the squared differences between green samples two
@@ -205,6 +219,20 @@ static constexpr unsigned int kBlueYMul = 29; /* 0.114 * 256 */
sumG += g; \
sumB += b; \
\
+ while (x >= zEndX && zCol + 1 < SwIspStats::kAwbZoneCols) { \
+ zRow[zCol].sumR += zSumR; \
+ zRow[zCol].sumG += zSumG; \
+ zRow[zCol].sumB += zSumB; \
+ zRow[zCol].count += zCount; \
+ zSumR = zSumG = zSumB = zCount = 0; \
+ zCol++; \
+ zEndX = (zCol + 1) * zLineLength / SwIspStats::kAwbZoneCols; \
+ } \
+ zSumR += r; \
+ zSumG += g; \
+ zSumB += b; \
+ zCount++; \
+ \
yVal = r * kRedYMul; \
yVal += g * kGreenYMul; \
yVal += b * kBlueYMul; \
@@ -227,6 +255,10 @@ static constexpr unsigned int kBlueYMul = 29; /* 0.114 * 256 */
stats.sum_.r() += sumR; \
stats.sum_.g() += sumG; \
stats.sum_.b() += sumB; \
+ zRow[zCol].sumR += zSumR; \
+ zRow[zCol].sumG += zSumG; \
+ zRow[zCol].sumB += zSumB; \
+ zRow[zCol].count += zCount; \
if (sharpCount) { \
const uint64_t meanG = sharpSumG / sharpCount; \
if (meanG) \
@@ -432,6 +464,7 @@ void SwStatsCpu::startFrame(uint32_t frame)
s.sum_ = RGB<uint64_t>({ 0, 0, 0 });
s.yHistogram.fill(0);
s.sharpness = 0;
+ s.awbZones.fill({ 0, 0, 0, 0 });
}
}
@@ -450,14 +483,26 @@ void SwStatsCpu::finishFrame(uint32_t frame, uint32_t bufferId)
sharedStats_->sum_ = RGB<uint64_t>({ 0, 0, 0 });
sharedStats_->sharpness = 0;
sharedStats_->yHistogram.fill(0);
+ sharedStats_->awbZones.fill({ 0, 0, 0, 0 });
for (const auto &s : stats_) {
sharedStats_->sharpness += s.sharpness;
sharedStats_->sum_ += s.sum_;
for (unsigned int j = 0; j < SwIspStats::kYHistogramSize; j++)
sharedStats_->yHistogram[j] += s.yHistogram[j];
+ for (unsigned int j = 0; j < sharedStats_->awbZones.size(); j++) {
+ sharedStats_->awbZones[j].sumR += s.awbZones[j].sumR;
+ sharedStats_->awbZones[j].sumG += s.awbZones[j].sumG;
+ sharedStats_->awbZones[j].sumB += s.awbZones[j].sumB;
+ sharedStats_->awbZones[j].count += s.awbZones[j].count;
+ }
}
sharedStats_->sum_ >>= sumShift_;
+ for (SwIspStats::AwbZone &zone : sharedStats_->awbZones) {
+ zone.sumR >>= sumShift_;
+ zone.sumG >>= sumShift_;
+ zone.sumB >>= sumShift_;
+ }
}
sharedStats_->valid = valid;
--
2.55.0

4
aports/temp/libcamera/APKBUILD
View file

@ -3,7 +3,7 @@ maintainer="Robert Mader <robert.mader@collabora.com>"
pkgname=libcamera
_pkgver=0.7.2
pkgver=9999$_pkgver
pkgrel=8
pkgrel=9
pkgdesc="Linux camera framework"
url="https://libcamera.org/"
arch="all"
@ -75,6 +75,7 @@ source="libcamera-v$_pkgver.tar.gz
0013-libcamera-software_isp-af-Add-AfMode-and-focus-witho.patch
0014-libcamera-software_isp-af-Re-baseline-the-scene-refe.patch
0015-ipa-simple-awb-Add-optional-white-point-locus-clamp-.patch
0016-ipa-simple-awb-Gate-the-gray-world-estimate-on-near-.patch
qcam.desktop
$_tuning_files
"
@ -203,6 +204,7 @@ c8ddc64ab943d9b215f4c997f2629e4403744c0ddb5de88cd42d384506a27c0762191eddd4d13559
dce81c2863ff6b9374115325f4af14ac5d396a6e2aee12f87d55a47d1d0bda860ae2af5a9d6885b126e20a12b7f3f40e49a2b9d738f94c04177bff6149da64e1 0013-libcamera-software_isp-af-Add-AfMode-and-focus-witho.patch
18ed6a03fbe3bcd750fac280831d1e62f2a2e35f76880535f2dc16ff196308512535e077ab0d8a923ad241090713edb53d1777e0dd6e5dcc10de495db1277052 0014-libcamera-software_isp-af-Re-baseline-the-scene-refe.patch
b01deba75c093ad4312ffa829ddbe5d745f2dfb199a3f00cdd52a8cb972fbe58bfb7cd94fa825b30d65047ac0c371ecf3ec0d0e356f090119d11086552f514be 0015-ipa-simple-awb-Add-optional-white-point-locus-clamp-.patch
a2dd2d360607c198022fd3353b5e9b4dcb0712abe3f0a6f60de138839b255aedea6a414760e29aa1cab82d0a84633755684d2f28ea0b15541241e51d7f5bd03e 0016-ipa-simple-awb-Gate-the-gray-world-estimate-on-near-.patch
22167a4eceb6d1b40b0b7c45fdf116c71684f5340de7f767535cb8e160ad9d2ae0f00cb3d461f73a344520a48a4641cf46226841d78bee06bfbfd2a91337f754 qcam.desktop
2ee566653b17d565d2c0de67cbe6ebad25df5aac6051cad78e6bae52016fa2ca0247e964d735ac776b3e0bd447ead9c4fcb26cd7629228cdba3ceb91b46f378f hi846.yaml
55dab9dcb9b1982143b9d63e4e51138c5bffd7c8f4a8e4e122750b3ad9d475446956bc297d9f564819ade4be75a575d06633a064e14f8ba3ef77986783c5f067 imx355.yaml