Opened 7 months ago

Last modified 7 weeks ago

#11019 new enhancement

Full-Image Averaging (Favg) denoise

Reported by: MasterQuestionable Owned by:
Priority: important Component: avfilter
Version: unspecified Keywords: denoise nlmeans
Cc: MasterQuestionable Blocked By:
Blocking: Reproduced by developer: no
Analyzed by developer: no

Description

͏    Follow-Up: https://trac.ffmpeg.org/wiki/denoise#favg

͏    Derived from my recent research on available denoise techniques.
͏    This seems to achieve theoretical best compressibility enhance: though unsure for subjective quality.
͏    .
͏    Some past research seems to prove the applicability:
͏    https://www.rationalqm.us/hiq/smoothhiq.html
͏    (sort of ͏"smartblur" derived Local Means?)

͏    The idea should work for audio too.

Change History (5)

comment:1 by MasterQuestionable, 7 months ago

͏    Theoretically the quality couldn't be much worse than weighted NLM. (and likely impossible to be worse than simple frequency clamp after FFT)
͏    But the algorithm seems to have significant speed benefit: potentially even faster than the FFT alone...

comment:2 by MasterQuestionable, 7 months ago

͏    The filter essentially only needs to handle 1 parameter:
͏    The difference threshold:
͏    |*| Valid range 0 ~ 1
͏    |*| May be expressed in percentage (e.g. 3.125%)
͏    |*| Or fraction (e.g. 8/255, 2,048/65,535)
͏    .
͏    Additional parameters shall just mean the threshold for each component (i.e. RGB, YUV etc.), or channel for audio:
͏    When undefined [1] assume the closest defined parameter. [2]
͏    Excessive parameters shall just be silently ignored. (support for arbitrary component configuration)
[1] "-" may be used instead of empty; for clarity.
͏    .
͏    It might be desirable to prepend a ͏"format" in the filter-chain (e.g. "format=rgb24,..."): to avoid the BGR alike havoc.

͏    Info about the interpreted fraction form should be reported: when the threshold not specified in corresponding fraction; unless the component/sample format is float derived.
͏    Warning should be given when the evaluation leads 0 or 1 (unless explicitly specified as [3]): as they are sort of special case.
[ [3]
͏    "0", "0%", "1", "100%";
͏    "${num}/${denum}", where ${num} = ( ${denum} | 0 ). ]

[ [2]
͏    E.g. "favg=3.125%:-:-:6.25%:::0"
͏    ${ /@[1-3]/ } = 3.125%
͏    ${ /@[4-6]/ } = 6.25%
͏    Rest = 0 ]

Last edited 7 months ago by MasterQuestionable (previous) (diff)

comment:3 by MasterQuestionable, 7 months ago

Keywords: nlmeans added
Priority: normalimportant

͏    Some low-effort testing seems to indicate the algorithm being able to give output similar (in some cases identical...) to ͏"nlmeans".

͏    This commit should be a good start point: (as copy)
͏    https://trac.ffmpeg.org/ticket/4910#comment:16
͏    .
͏    The algorithm is essentially NLM without the patch matching weighting.
͏    (and optimized to avoid pointless work)

͏    See also: https://www.mail-archive.com/search?l=ffmpeg-devel@ffmpeg.org&q=subject:%22nlmeans%22&o=oldest&f=1

͏    [ Quote Clément Bœsch @ CE 2018-05-07 17:24:46 UTC:
https://www.mail-archive.com/ffmpeg-devel@ffmpeg.org/msg69473.html
͏    ... , the filter badly needs some performance improvements.
͏    Also, any suggestion on how not to make it spend 80% of the time in "nlmeans_slice" welcome. ]
<^>    Never considered to drop the patch matching outright..?


͏    Further analysis also indicated the patch matching of "nlmeans" might actually spoil the image:
͏    Without really denoise but instead introduced noise...
͏    (unjustified blurriness near originally hard color edges)

Last edited 7 weeks ago by MasterQuestionable (previous) (diff)

comment:4 by MasterQuestionable, 5 months ago

͏    Another interesting property of the algorithm seems to be idempotence:
͏    I.e. repeatedly applying which with equal or less strength seems to keep output identical.

͏    E.g.
͏    19, 20 * n, 25, 30 * n
͏    ; where " * n" means countless many.
͏    .
͏    Possible outcomes:
͏    (given threshold of 5)
͏    |1| ~ 20 * n, ~ 30 * n
͏    |2| ~ 20 * n, 30 * n
͏    |3| 19, 25 * n

͏    Making which a perfect substitute of JPEG alike compression schemes.
͏    (no further re-encoding loss)


͏    Per above observation, it's likely also beneficial to make the averaging process:
͏    Start from the most frequently appearing values (per component/channel).

͏    Such shall give more reasonable averaging outcome.

͏    The original variant might also be of interest due to speed advantage.
͏    Which could be named as "sfavg" (sFavg: Speedy Favg).

Last edited 3 months ago by MasterQuestionable (previous) (diff)

comment:5 by MasterQuestionable, 4 months ago

͏    Though lack sufficient evidence support:
͏    The described approach appears to be the theoretical optimal method for denoise..? (base method for all kinds)
͏    Anything further would only involve selecting more suitable strength for each exact sample.

͏    Which introduces no method noise (like JPEG, H.264 etc. did).
͏    And considering what makes denoise: the algorithm just seems to achieve what theoretically best achievable? (actually more appropriate than typical "weighted" NLM...)
͏    Plus the idempotence...

͏    Probably a revolution to all lossy compression schemes?

Note: See TracTickets for help on using tickets.