Catcrafts/fp6-img
Watch
2
0
Fork
You've already forked fp6-img
 0
Code Issues Pull requests Projects Releases 1 Packages Wiki Activity Actions

libcamera: stabilize AWB with a white point locus clamp + damping (r8)

Browse source 
The softISP AWB is pure gray world, so panning swings ColourGains with
the frame average (white wall renders green when the frame is
warm-dominated, dark desk renders red when cool-dominated) and the CCM,
interpolated over a CT estimated from those gains, amplifies and
quantizes the swings.

Patch 0015 (ours, original work) adds optional per-sensor Awb tuning to
the simple IPA: a piecewise-linear locus of calibrated illuminant white
points in gain space that the gray-world estimate is clamped to
(clampMargin), a per-stats-frame EMA on the applied gains (damping)
with a fast-converge path for persistent illuminant changes, and CT
interpolated from the applied gains' position along the locus - on the
same scale as the tuning file's ccms table - instead of the generic
estimateCCT(). Tuning files without whitePoints keep the previous
behaviour exactly, so every other sensor is unaffected.

imx896.yaml gains the locus from the stock tuning blob's 10-point
per-illuminant AWB calibration (white points and their exactly
reciprocal gain triplets, cross-validated): the 7 on-locus points as
nodes, the 3 fluorescent points admitted via clampMargin 0.35. The
5000 K label is the blob's own proven anchor; 2856 K (A) is locked by
the fluorescent trio projecting onto the 2856-5000 chord at ~3960 K
(TL84); the rest are geometry-derived approximations.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
This commit is contained in:
Jorijn van der Graaf 2026-08-17 03:24:56 +02:00
commit 48cedac3c3
3 changed files with 437 additions and 2 deletions
Download patch file Download diff file Expand all files Collapse all files

408
aports/temp/libcamera/0015-ipa-simple-awb-Add-optional-white-point-locus-clamp-.patch Normal file
View file

@ -0,0 +1,408 @@
From 62fc9d05460aa0ad5e1e4c8f81ae966796c066c9 Mon Sep 17 00:00:00 2001
From: Jorijn van der Graaf <jorijnvdgraaf@catcrafts.net>
Date: Mon, 17 Aug 2026 03:23:30 +0200
Subject: [PATCH] ipa: simple: awb: Add optional white point locus clamp and
damping
The simple IPA's AWB is pure gray world: every stats frame the applied
gains become the frame-average G/R and G/B ratios of that frame. Two
artifacts follow on real hardware. Panning swings the gains with the
scene composition - a warm-dominated frame renders a white wall green,
a cool-dominated one renders a dark desk red - and the CCM, which is
interpolated over a colour temperature estimated from those same gains,
amplifies and quantizes each swing.
Add optional per-sensor tuning for the Awb algorithm:
- Awb:
whitePoints: # ordered by strictly ascending ct
- { ct: 2856, gains: [ 1.378, 2.769 ] } # [ R gain, B gain ]
- { ct: 5000, gains: [ 2.037, 1.809 ] }
clampMargin: 0.35 # max distance from the locus in gain space
damping: 0.8 # per-stats-frame EMA weight of the old gains
whitePoints defines a piecewise-linear locus of calibrated illuminant
white points in (R gain, B gain) space. The gray-world estimate is
computed as before, then clamped to within clampMargin of the locus:
scene colours can no longer pull the white balance arbitrarily far from
colours a real illuminant would produce, while off-locus (greenish,
e.g. fluorescent) illuminants within the margin still get corrected.
The applied gains follow the clamped estimate through a per-stats-frame
exponential moving average so composition changes fade in smoothly; a
large difference persisting for several stats frames is a real
illuminant change and halves the damping so scene cuts converge quickly
instead of crawling.
The colour temperature is no longer estimated with the generic
estimateCCT() matrix but interpolated from the applied gains' position
along the locus, so the Ccm algorithm always picks a matrix consistent
with the applied white balance, on the same ct scale the tuning file's
ccms table uses.
A tuning file without whitePoints keeps the previous behaviour exactly.
Signed-off-by: Jorijn van der Graaf <jorijnvdgraaf@catcrafts.net>
---
src/ipa/simple/algorithms/awb.cpp | 240 ++++++++++++++++++++++++++++--
src/ipa/simple/algorithms/awb.h | 30 ++++
2 files changed, 257 insertions(+), 13 deletions(-)
diff --git a/src/ipa/simple/algorithms/awb.cpp b/src/ipa/simple/algorithms/awb.cpp
index 05155c8..83dcece 100644
--- a/src/ipa/simple/algorithms/awb.cpp
+++ b/src/ipa/simple/algorithms/awb.cpp
@@ -1,34 +1,160 @@
/* SPDX-License-Identifier: LGPL-2.1-or-later */
/*
* Copyright (C) 2024-2026 Red Hat Inc.
+ * Copyright (C) 2026 Jorijn van der Graaf
*
* Auto white balance
*/
#include "awb.h"
+#include <algorithm>
+#include <cmath>
+#include <limits>
#include <numeric>
+#include <optional>
#include <stdint.h>
#include <libcamera/base/log.h>
#include <libcamera/control_ids.h>
+#include "libcamera/internal/value_node.h"
+#include "libcamera/internal/vector.h"
+
#include "libipa/colours.h"
#include "simple/ipa_context.h"
+namespace {
+
+constexpr double kDefaultClampMargin = 0.35;
+constexpr double kDefaultDamping = 0.8;
+
+/*
+ * When the damped gains stay farther than kFastConvergeDistance (in gain
+ * space) from the clamped gray-world target for more than
+ * kFastConvergeFrames consecutive stats frames, the difference is a real
+ * illuminant change rather than scene-composition noise: halve the damping
+ * so the change converges quickly instead of crawling.
+ */
+constexpr double kFastConvergeDistance = 0.15;
+constexpr unsigned int kFastConvergeFrames = 3;
+
+/* Start from the white point closest to daylight. */
+constexpr double kInitialCT = 5000.0;
+
+} /* namespace */
+
namespace libcamera {
LOG_DEFINE_CATEGORY(IPASoftAwb)
namespace ipa::soft::algorithms {
+/*
+ * Optional per-sensor tuning:
+ *
+ * - Awb:
+ * whitePoints: # ordered by strictly ascending ct
+ * - { ct: 2856, gains: [ 1.378, 2.769 ] } # [ R gain, B gain ]
+ * - { ct: 5000, gains: [ 2.037, 1.809 ] }
+ * clampMargin: 0.35 # max distance from the locus in gain space
+ * damping: 0.8 # per-stats-frame EMA weight of the old gains
+ *
+ * whitePoints defines a piecewise-linear locus of calibrated illuminant
+ * white points in (R gain, B gain) space. The gray-world estimate is
+ * clamped to within clampMargin of the locus, bounding how far scene
+ * colours can pull the white balance while still allowing off-locus
+ * (greenish, e.g. fluorescent) illuminants. The applied gains follow the
+ * clamped estimate through an exponential moving average, and the colour
+ * temperature fed to the Ccm algorithm is interpolated from the applied
+ * gains' position along the locus. Without whitePoints the plain
+ * gray-world algorithm runs unchanged.
+ */
+int Awb::init([[maybe_unused]] IPAContext &context, const ValueNode &tuningData)
+{
+ const ValueNode &wps = tuningData["whitePoints"];
+ if (wps.isEmpty())
+ return 0;
+
+ if (!wps.isList() || wps.size() < 2) {
+ LOG(IPASoftAwb, Error)
+ << "whitePoints must be a list of at least two points";
+ return -EINVAL;
+ }
+
+ for (const ValueNode &node : wps.asList()) {
+ std::optional<double> ct = node["ct"].get<double>();
+ std::optional<Vector<double, 2>> gains =
+ node["gains"].get<Vector<double, 2>>();
+ if (!ct || !gains || (*gains)[0] <= 0.0 || (*gains)[1] <= 0.0) {
+ LOG(IPASoftAwb, Error)
+ << "whitePoints entries need a ct and positive gains [ R, B ]";
+ whitePoints_.clear();
+ return -EINVAL;
+ }
+ if (!whitePoints_.empty()) {
+ const WhitePoint &prev = whitePoints_.back();
+ if (*ct <= prev.ct) {
+ LOG(IPASoftAwb, Error)
+ << "whitePoints must be ordered by strictly ascending ct";
+ whitePoints_.clear();
+ return -EINVAL;
+ }
+ if ((*gains)[0] == prev.rGain && (*gains)[1] == prev.bGain) {
+ LOG(IPASoftAwb, Error)
+ << "consecutive whitePoints must differ in gains";
+ whitePoints_.clear();
+ return -EINVAL;
+ }
+ }
+ whitePoints_.push_back({ *ct, (*gains)[0], (*gains)[1] });
+ }
+
+ clampMargin_ = tuningData["clampMargin"].get<double>(kDefaultClampMargin);
+ damping_ = tuningData["damping"].get<double>(kDefaultDamping);
+ if (clampMargin_ <= 0.0 || damping_ < 0.0 || damping_ >= 1.0) {
+ LOG(IPASoftAwb, Error)
+ << "clampMargin must be positive and damping within [0, 1)";
+ whitePoints_.clear();
+ return -EINVAL;
+ }
+
+ LOG(IPASoftAwb, Info)
+ << "White point locus: " << whitePoints_.size() << " points, "
+ << whitePoints_.front().ct << "-" << whitePoints_.back().ct
+ << " K, clamp margin " << clampMargin_
+ << ", damping " << damping_;
+
+ return 0;
+}
+
int Awb::configure(IPAContext &context,
[[maybe_unused]] const IPAConfigInfo &configInfo)
{
auto &gains = context.activeState.awb.gains;
gains = { { 1.0, 1.0, 1.0 } };
+ fastFrames_ = 0;
+
+ if (whitePoints_.empty())
+ return 0;
+
+ /*
+ * Start from the calibrated white point closest to daylight rather
+ * than from unity gains, so the first frames are roughly plausible
+ * while the loop converges.
+ */
+ const WhitePoint *initial = &whitePoints_.front();
+ for (const WhitePoint &wp : whitePoints_) {
+ if (std::abs(wp.ct - kInitialCT) < std::abs(initial->ct - kInitialCT))
+ initial = &wp;
+ }
+ gains = { { static_cast<float>(initial->rGain), 1.0,
+ static_cast<float>(initial->bGain) } };
+ context.activeState.awb.temperatureK =
+ static_cast<unsigned int>(std::lround(initial->ct));
+
return 0;
}
@@ -43,6 +169,39 @@ void Awb::prepare(IPAContext &context,
params->gains = gains;
}
+/*
+ * Find the closest point on the piecewise-linear white point locus, and
+ * the colour temperature interpolated along the containing segment.
+ */
+Awb::LocusPosition Awb::closestOnLocus(double rGain, double bGain) const
+{
+ LocusPosition best{};
+ double bestDist2 = std::numeric_limits<double>::max();
+
+ for (unsigned int i = 0; i + 1 < whitePoints_.size(); i++) {
+ const WhitePoint &p0 = whitePoints_[i];
+ const WhitePoint &p1 = whitePoints_[i + 1];
+ const double dr = p1.rGain - p0.rGain;
+ const double db = p1.bGain - p0.bGain;
+ const double len2 = dr * dr + db * db;
+ double t = ((rGain - p0.rGain) * dr + (bGain - p0.bGain) * db) / len2;
+ t = std::clamp(t, 0.0, 1.0);
+ const double qr = p0.rGain + t * dr;
+ const double qb = p0.bGain + t * db;
+ const double dist2 = (rGain - qr) * (rGain - qr) +
+ (bGain - qb) * (bGain - qb);
+ if (dist2 < bestDist2) {
+ bestDist2 = dist2;
+ best.rGain = qr;
+ best.bGain = qb;
+ best.ct = p0.ct + t * (p1.ct - p0.ct);
+ }
+ }
+
+ best.distance = std::sqrt(bestDist2);
+ return best;
+}
+
void Awb::process(IPAContext &context,
[[maybe_unused]] const uint32_t frame,
IPAFrameContext &frameContext,
@@ -73,24 +232,79 @@ void Awb::process(IPAContext &context,
*/
const RGB<uint64_t> sum = stats->sum_.max(offset + minValid) - offset;
+ auto &gains = context.activeState.awb.gains;
+
+ if (whitePoints_.empty()) {
+ /*
+ * Calculate red and blue gains for AWB.
+ * Clamp max gain at 4.0, this also avoids 0 division.
+ */
+ gains = { {
+ sum.r() <= sum.g() / 4 ? 4.0f : static_cast<float>(sum.g()) / sum.r(),
+ 1.0,
+ sum.b() <= sum.g() / 4 ? 4.0f : static_cast<float>(sum.g()) / sum.b(),
+ } };
+
+ RGB<double> rgbGains{ { 1 / gains.r(), 1 / gains.g(), 1 / gains.b() } };
+ context.activeState.awb.temperatureK = estimateCCT(rgbGains);
+ metadata.set(controls::ColourTemperature,
+ context.activeState.awb.temperatureK);
+
+ LOG(IPASoftAwb, Debug)
+ << "gain R/B: " << gains << "; temperature: "
+ << context.activeState.awb.temperatureK;
+
+ return;
+ }
+
/*
- * Calculate red and blue gains for AWB.
- * Clamp max gain at 4.0, this also avoids 0 division.
+ * Gray-world estimate. The sums are at least minValid, so the
+ * divisions are safe; the locus clamp below bounds the result.
*/
- auto &gains = context.activeState.awb.gains;
- gains = { {
- sum.r() <= sum.g() / 4 ? 4.0f : static_cast<float>(sum.g()) / sum.r(),
- 1.0,
- sum.b() <= sum.g() / 4 ? 4.0f : static_cast<float>(sum.g()) / sum.b(),
- } };
+ double rTarget = static_cast<double>(sum.g()) / sum.r();
+ double bTarget = static_cast<double>(sum.g()) / sum.b();
+
+ /* Clamp the estimate to within clampMargin_ of the locus. */
+ const LocusPosition projection = closestOnLocus(rTarget, bTarget);
+ if (projection.distance > clampMargin_) {
+ const double f = clampMargin_ / projection.distance;
+ rTarget = projection.rGain + (rTarget - projection.rGain) * f;
+ bTarget = projection.bGain + (bTarget - projection.bGain) * f;
+ }
- RGB<double> rgbGains{ { 1 / gains.r(), 1 / gains.g(), 1 / gains.b() } };
- context.activeState.awb.temperatureK = estimateCCT(rgbGains);
- metadata.set(controls::ColourTemperature, context.activeState.awb.temperatureK);
+ /*
+ * Damp the applied gains toward the clamped target so that changes
+ * of the frame composition (e.g. panning) do not swing the colours,
+ * converging faster when the difference is large and persistent.
+ */
+ const double err = std::hypot(rTarget - gains.r(), bTarget - gains.b());
+ if (err > kFastConvergeDistance)
+ fastFrames_++;
+ else
+ fastFrames_ = 0;
+ double damping = damping_;
+ if (fastFrames_ > kFastConvergeFrames)
+ damping /= 2.0;
+
+ const double rNew = damping * gains.r() + (1.0 - damping) * rTarget;
+ const double bNew = damping * gains.b() + (1.0 - damping) * bTarget;
+ gains = { { static_cast<float>(rNew), 1.0, static_cast<float>(bNew) } };
+
+ /*
+ * The colour temperature the Ccm algorithm consumes follows from
+ * where the applied gains sit along the locus, so the matrix always
+ * matches the applied white balance.
+ */
+ const LocusPosition applied = closestOnLocus(rNew, bNew);
+ context.activeState.awb.temperatureK =
+ static_cast<unsigned int>(std::lround(applied.ct));
+ metadata.set(controls::ColourTemperature,
+ context.activeState.awb.temperatureK);
LOG(IPASoftAwb, Debug)
- << "gain R/B: " << gains << "; temperature: "
- << context.activeState.awb.temperatureK;
+ << "gain R/B: " << gains
+ << "; locus distance: " << projection.distance
+ << "; temperature: " << context.activeState.awb.temperatureK;
}
REGISTER_IPA_ALGORITHM(Awb, "Awb")
diff --git a/src/ipa/simple/algorithms/awb.h b/src/ipa/simple/algorithms/awb.h
index ad993f3..ef4e4f3 100644
--- a/src/ipa/simple/algorithms/awb.h
+++ b/src/ipa/simple/algorithms/awb.h
@@ -1,12 +1,15 @@
/* SPDX-License-Identifier: LGPL-2.1-or-later */
/*
* Copyright (C) 2024-2025 Red Hat Inc.
+ * Copyright (C) 2026 Jorijn van der Graaf
*
* Auto white balance
*/
#pragma once
+#include <vector>
+
#include "algorithm.h"
namespace libcamera {
@@ -19,6 +22,7 @@ public:
Awb() = default;
~Awb() = default;
+ int init(IPAContext &context, const ValueNode &tuningData) override;
int configure(IPAContext &context, const IPAConfigInfo &configInfo) override;
void prepare(IPAContext &context,
const uint32_t frame,
@@ -29,6 +33,32 @@ public:
IPAFrameContext &frameContext,
const SwIspStats *stats,
ControlList &metadata) override;
+
+private:
+ struct WhitePoint {
+ double ct;
+ double rGain;
+ double bGain;
+ };
+
+ struct LocusPosition {
+ double rGain;
+ double bGain;
+ double ct;
+ double distance;
+ };
+
+ LocusPosition closestOnLocus(double rGain, double bGain) const;
+
+ /*
+ * Calibrated white point locus in (R gain, B gain) space, ordered by
+ * strictly ascending colour temperature. Empty when the tuning file
+ * provides no white points; the plain gray-world algorithm runs then.
+ */
+ std::vector<WhitePoint> whitePoints_;
+ double clampMargin_ = 0.0;
+ double damping_ = 0.0;
+ unsigned int fastFrames_ = 0;
};
} /* namespace ipa::soft::algorithms */

6
aports/temp/libcamera/APKBUILD
View file

@ -3,7 +3,7 @@ maintainer="Robert Mader <robert.mader@collabora.com>"
pkgname=libcamera
_pkgver=0.7.2
pkgver=9999$_pkgver
pkgrel=7
pkgrel=8
pkgdesc="Linux camera framework"
url="https://libcamera.org/"
arch="all"
@ -72,6 +72,7 @@ source="https://gitlab.freedesktop.org/camera/libcamera/-/archive/v$_pkgver/libc
0012-libcamera-software_isp-af-Harden-the-lens-control-ha.patch
0013-libcamera-software_isp-af-Add-AfMode-and-focus-witho.patch
0014-libcamera-software_isp-af-Re-baseline-the-scene-refe.patch
0015-ipa-simple-awb-Add-optional-white-point-locus-clamp-.patch
qcam.desktop
$_tuning_files
"
@ -199,6 +200,7 @@ c8ddc64ab943d9b215f4c997f2629e4403744c0ddb5de88cd42d384506a27c0762191eddd4d13559
8734e27966bd1bc5a4b0324a3d8a3ee4ab371f6d1536500b2bb44d49bde874c3fd6f244312332226c352e4716ba12fb72bf51e64b9e97cf6fa646aa6dfef3277 0012-libcamera-software_isp-af-Harden-the-lens-control-ha.patch
dce81c2863ff6b9374115325f4af14ac5d396a6e2aee12f87d55a47d1d0bda860ae2af5a9d6885b126e20a12b7f3f40e49a2b9d738f94c04177bff6149da64e1 0013-libcamera-software_isp-af-Add-AfMode-and-focus-witho.patch
18ed6a03fbe3bcd750fac280831d1e62f2a2e35f76880535f2dc16ff196308512535e077ab0d8a923ad241090713edb53d1777e0dd6e5dcc10de495db1277052 0014-libcamera-software_isp-af-Re-baseline-the-scene-refe.patch
b01deba75c093ad4312ffa829ddbe5d745f2dfb199a3f00cdd52a8cb972fbe58bfb7cd94fa825b30d65047ac0c371ecf3ec0d0e356f090119d11086552f514be 0015-ipa-simple-awb-Add-optional-white-point-locus-clamp-.patch
22167a4eceb6d1b40b0b7c45fdf116c71684f5340de7f767535cb8e160ad9d2ae0f00cb3d461f73a344520a48a4641cf46226841d78bee06bfbfd2a91337f754 qcam.desktop
2ee566653b17d565d2c0de67cbe6ebad25df5aac6051cad78e6bae52016fa2ca0247e964d735ac776b3e0bd447ead9c4fcb26cd7629228cdba3ceb91b46f378f hi846.yaml
55dab9dcb9b1982143b9d63e4e51138c5bffd7c8f4a8e4e122750b3ad9d475446956bc297d9f564819ade4be75a575d06633a064e14f8ba3ef77986783c5f067 imx355.yaml
@ -207,7 +209,7 @@ dce81c2863ff6b9374115325f4af14ac5d396a6e2aee12f87d55a47d1d0bda860ae2af5a9d6885b1
55dab9dcb9b1982143b9d63e4e51138c5bffd7c8f4a8e4e122750b3ad9d475446956bc297d9f564819ade4be75a575d06633a064e14f8ba3ef77986783c5f067 imx376.yaml
2ee566653b17d565d2c0de67cbe6ebad25df5aac6051cad78e6bae52016fa2ca0247e964d735ac776b3e0bd447ead9c4fcb26cd7629228cdba3ceb91b46f378f imx519.yaml
55dab9dcb9b1982143b9d63e4e51138c5bffd7c8f4a8e4e122750b3ad9d475446956bc297d9f564819ade4be75a575d06633a064e14f8ba3ef77986783c5f067 imx858.yaml
cbb6204657fdb4bc35e9ee686200900291c8bb74678d3f19f8ed30120e5d939836534f8724726c58e8b7aa2eb99e99edace96350b11472fbbf4878cc1025c3f5 imx896.yaml
918bc5fc2035ba32af617a9f00a98dda1b8272cab04d7cea12bdf26b7e0d9811a8c8c2061191e00174d01a1ad991a3327bf418ce89dfb3610d06787f14389fa0 imx896.yaml
2ee566653b17d565d2c0de67cbe6ebad25df5aac6051cad78e6bae52016fa2ca0247e964d735ac776b3e0bd447ead9c4fcb26cd7629228cdba3ceb91b46f378f s5k3l6xx.yaml
d59020e62520ee622282c85ce7bcdfe453c99becf48cc1005c5f8a4bb8b2511b19c89f38fb3c994e7c6a79d1f84e2ed5f5a09b4c5bdc1dc4a44b071dbccf0d44 s5kjn1.yaml
"

25
aports/temp/libcamera/imx896.yaml
View file

@ -19,6 +19,21 @@
# dynamic estimate ran at 48, leaving a 16-code residual the CCM amplified
# into colour casts in dark regions.
#
# Awb whitePoints: the 10-point per-illuminant AWB calibration from the
# same blob (white points @0x1c69ed, gain triplets @0x1c2dba — exact
# reciprocals of each other, cross-validated to float precision). The
# locus below carries the 7 on-locus points as (R gain, B gain), warm to
# cool. The 3 fluorescent points (idx 4-6) are deliberately NOT locus
# nodes: they sit ~0.33 gain units off the A-D50 chord (green side) and
# would zigzag the locus; clampMargin 0.35 admits them instead. CT
# labels: 5000 K is PROVEN (the blob's default gain + CCT anchor at
# index 3); 2856 K (illuminant A) is locked by two independent checks —
# sensor-typical A ratios AND the fluorescent trio projecting onto the
# 2856-5000 chord at ~3960 K, i.e. exactly TL84; the remaining labels
# are geometry-derived approximations (the blob stores no CCT list).
# Same ct scale as the ccms table below. damping = per-stats-frame EMA
# (stats run every 4th frame), ~1 s settling at 30 fps.
#
# PROVENANCE / LICENSE: the matrices are numeric calibration data read out
# of the proprietary vendor blob (measured spectral response of this
# sensor, not authored expression). Shipping decided 2026-08-17 by the
@ -32,6 +47,16 @@ algorithms:
- BlackLevel:
blackLevel: 4096
- Awb:
whitePoints:
- { ct: 1900, gains: [ 0.878997, 4.771635 ] }
- { ct: 2300, gains: [ 1.119141, 3.340533 ] }
- { ct: 2856, gains: [ 1.377957, 2.769483 ] }
- { ct: 5000, gains: [ 2.037341, 1.809127 ] }
- { ct: 6504, gains: [ 2.367133, 1.574040 ] }
- { ct: 7504, gains: [ 2.741927, 1.357219 ] }
- { ct: 9500, gains: [ 4.508486, 1.012164 ] }
clampMargin: 0.35
damping: 0.8
- Ccm:
ccms:
- ct: 2800