Skip to content
Draft
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
38 changes: 38 additions & 0 deletions docs/configuration.md
Original file line number Diff line number Diff line change
Expand Up @@ -2129,6 +2129,44 @@ editing the `conf` file in a text editor. Use the examples as reference.
</tr>
</table>

### chroma_supersample

<table>
<tr>
<td>Description</td>
<td colspan="2">
When the stream resolution is an exact, even integer multiple of the capture resolution (2x, 4x, ...) in
both axes, use nearest-neighbour (point) sampling for the capture-to-stream upscale instead of the
default bilinear filter.
@note{This makes 4:2:0 chroma subsampling exact: every captured pixel is replicated into an identical
NxN block, so the encoder's 2x2 chroma averaging reproduces the capture's own chroma bit-for-bit, prior
to quantisation. This is a practical alternative to 4:4:4 encoding on GPUs whose hardware encoder has no
4:4:4 profile at all (e.g. AMD VCN, Intel Quick Sync). See
[Moonlight issue #1671](https://github.com/moonlight-stream/moonlight-qt/issues/1671) for the
protocol-level version of the same idea; this option makes the existing supersampling workaround
exact without any client-side changes.}
@note{The client (Moonlight) must present the stream at exactly 1/N of the stream resolution for the
property to hold end to end — fullscreen playback is the reliable way to get this; a window with
decorations, or any non-integer client-side scaling, silently breaks it.}
@note{Only applies on Linux, where the capture-to-stream scale is a single GL blit. When the geometry
does not qualify (non-integer or odd ratio, mismatched axes), Sunshine logs a warning and falls back to
the existing bilinear scaling — the stream still works, just without the exactness property.}
</td>
</tr>
<tr>
<td>Default</td>
<td colspan="2">@code{}
disabled
@endcode</td>
</tr>
<tr>
<td>Example</td>
<td colspan="2">@code{}
chroma_supersample = enabled
@endcode</td>
</tr>
</table>

### capture

<table>
Expand Down
4 changes: 4 additions & 0 deletions src/config.cpp
Original file line number Diff line number Diff line change
Expand Up @@ -772,6 +772,8 @@ namespace config {
2, // vk.rc_mode (default: cbr)
},

false, // chroma_supersample

{}, // capture
{}, // encoder
{}, // adapter_name
Expand Down Expand Up @@ -1659,6 +1661,8 @@ namespace config {
int_f(vars, "vk_tune", video.vk.tune);
int_f(vars, "vk_rc_mode", video.vk.rc_mode);

bool_f(vars, "chroma_supersample", video.chroma_supersample);

string_f(vars, "capture", video.capture);
string_f(vars, "encoder", video.encoder);
string_f(vars, "adapter_name", video.adapter_name);
Expand Down
2 changes: 2 additions & 0 deletions src/config.h
Original file line number Diff line number Diff line change
Expand Up @@ -118,6 +118,8 @@ namespace config {
int rc_mode; // 0=driver, 1=cqp, 2=cbr, 4=vbr
} vk; ///< Vulkan encoder options.

bool chroma_supersample; ///< Use point sampling when the stream resolution is an exact, even integer multiple of the capture resolution.

std::string capture; ///< Capture backend name selected by configuration.
std::string encoder; ///< Encoder backend name selected by configuration.
std::string adapter_name; ///< Display adapter name selected in configuration.
Expand Down
97 changes: 94 additions & 3 deletions src/platform/linux/graphics.cpp
Original file line number Diff line number Diff line change
Expand Up @@ -7,6 +7,7 @@

// local includes
#include "graphics.h"
#include "src/config.h"
#include "src/file_handler.h"
#include "src/logging.h"
#include "src/video.h"
Expand Down Expand Up @@ -994,6 +995,75 @@ namespace egl {
return 0;
}

int supersample_factor(int in_width, int in_height, int out_width, int out_height) {
if (in_width <= 0 || in_height <= 0) {
return 0;
}

const int factor = out_width / in_width;
if (factor < 2) {
return 0;
}

// Both axes must scale by the same exact integer. Anything else maps a captured pixel
// onto a fractional number of stream pixels, which point sampling would alias.
if (out_width != factor * in_width || out_height != factor * in_height) {
return 0;
}

// The factor must also be even. 4:2:0 chroma is a box average of a 2x2 window of stream
// pixels; that window lands entirely inside one replicated source block only when the
// block size is even, so every 2x2 window is either fully inside one block or fully
// straddles two equal-valued blocks - both exact. An odd factor puts some windows half in
// one block and half in the next, blending two different source pixels' chroma; measured
// empirically to be *worse* than bilinear at 3x, not just merely inexact.
if (factor & 1) {
return 0;
}

return factor;
}

/**
* @brief Select point sampling for the capture-to-stream upscale when it is exactly integral.
*
* Leaves sws.supersample at 0 - and therefore the pipeline on its default bilinear filter -
* unless `chroma_supersample` is enabled and the geometry supports exact pixel replication.
*
* @param sws Software-scaling pipeline to configure.
* @param is_yuv444 Whether the destination uses three full-resolution planes.
*/
void configure_supersampling(sws_t &sws, bool is_yuv444) {
if (!config::video.chroma_supersample) {
return;
}

const int factor = supersample_factor(sws.in_width, sws.in_height, sws.out_width, sws.out_height);
if (!factor) {
BOOST_LOG(warning)
<< "chroma_supersample: stream resolution ["sv << sws.out_width << 'x' << sws.out_height
<< "] is not an exact, even integer multiple (2x, 4x, ...) of the capture resolution ["sv
<< sws.in_width << 'x' << sws.in_height << "]; keeping bilinear scaling"sv;
return;
}

// The chroma planes of a 4:2:0 target render into a viewport at exactly half the luma
// offset. An odd offset truncates, shifting the chroma grid half a captured pixel away
// from the luma blocks it belongs to.
if (!is_yuv444 && ((sws.offsetX & 1) || (sws.offsetY & 1))) {
BOOST_LOG(warning)
<< "chroma_supersample: letterbox offset ["sv << sws.offsetX << ',' << sws.offsetY
<< "] is odd; keeping bilinear scaling"sv;
return;
}

sws.supersample = factor;

BOOST_LOG(info)
<< "chroma_supersample: point sampling ["sv << sws.in_width << 'x' << sws.in_height
<< "] -> ["sv << sws.out_width << 'x' << sws.out_height << "] ("sv << factor << "x)"sv;
}

std::optional<sws_t> sws_t::make_nv12(int in_width, int in_height, int out_width, int out_height, gl::tex_t &&tex) {
sws_t sws;

Expand All @@ -1017,7 +1087,14 @@ namespace egl {
sws.offsetX = offsetX_f;
sws.offsetY = offsetY_f;

auto width_i = 1.0f / sws.out_width;
configure_supersampling(sws, false);

// ConvertUV.frag averages two horizontal taps, `width_i` apart, to place the chroma
// sample on the left-sited position H.264/HEVC assume by default. Under point sampling
// both taps already resolve to the same captured pixel - every position inside an NxN
// block holds the same value - so the offset is dropped to keep that exact instead of
// relying on the two taps rounding to the same texel.
auto width_i = sws.supersample ? 0.0f : 1.0f / sws.out_width;

{
constexpr std::array<const char *, 5> sources {{
Expand Down Expand Up @@ -1123,6 +1200,8 @@ namespace egl {
sws.offsetX = offsetX_f;
sws.offsetY = offsetY_f;

configure_supersampling(sws, true);

{
constexpr std::array<const char *, 5> sources {{
SUNSHINE_SHADERS_DIR "/Scene.vert",
Expand Down Expand Up @@ -1349,9 +1428,21 @@ namespace egl {
return 0;
}

int sws_t::convert_nv12(gl::frame_buf_t &fb) {
void sws_t::bind_source_texture() {
gl::ctx.BindTexture(GL_TEXTURE_2D, loaded_texture);

if (supersample) {
// Set here rather than in gl::tex_t::make(), which backs every texture in the GL path
// including the cursor. The source may also be a texture imported from a DMA-BUF and
// owned elsewhere, so this is the only point that reliably covers both cases.
gl::ctx.TexParameteri(GL_TEXTURE_2D, GL_TEXTURE_MIN_FILTER, GL_NEAREST);
gl::ctx.TexParameteri(GL_TEXTURE_2D, GL_TEXTURE_MAG_FILTER, GL_NEAREST);
}
}

int sws_t::convert_nv12(gl::frame_buf_t &fb) {
bind_source_texture();

GLenum attachments[] {
GL_COLOR_ATTACHMENT0,
GL_COLOR_ATTACHMENT1
Expand All @@ -1372,7 +1463,7 @@ namespace egl {
}

int sws_t::convert_yuv444(gl::frame_buf_t &fb) {
gl::ctx.BindTexture(GL_TEXTURE_2D, loaded_texture);
bind_source_texture();

GLenum attachments[] {
GL_COLOR_ATTACHMENT0,
Expand Down
31 changes: 31 additions & 0 deletions src/platform/linux/graphics.h
Original file line number Diff line number Diff line change
Expand Up @@ -634,6 +634,26 @@ namespace egl {
std::optional<uint32_t> pw_flags; ///< PipeWire frame flags reported with the buffer.
};

/**
* @brief Determine the exact integer capture-to-stream upscale factor.
*
* Point (nearest-neighbour) upscaling only replicates each captured pixel into an
* identical NxN block when the stream resolution is an exact integer multiple of the
* capture resolution in both axes. The factor must additionally be even: 4:2:0 chroma is
* a box average of a 2x2 window of stream pixels, and that window only ever falls fully
* inside one replicated block (or fully spans two identical ones) when the block size is
* even. An odd factor blends two different source pixels' chroma in some windows -
* measured to be worse than the existing bilinear scaling, not just merely inexact. Any
* ratio that fails either test must keep the default bilinear filter.
*
* @param in_width Capture width in pixels.
* @param in_height Capture height in pixels.
* @param out_width Aspect-corrected stream width in pixels.
* @param out_height Aspect-corrected stream height in pixels.
* @return The upscale factor when it is an even integer of at least 2 and identical in both axes, otherwise 0.
*/
int supersample_factor(int in_width, int in_height, int out_width, int out_height);

/**
* @brief EGL/OpenGL scaler and colorspace conversion pipeline.
*/
Expand Down Expand Up @@ -719,6 +739,14 @@ namespace egl {
*/
int blank(gl::frame_buf_t &fb, int offsetX_, int offsetY_, int width, int height, bool is_yuv444);

/**
* @brief Bind the loaded source texture and select the filter used by the scaling blit.
*
* Applies GL_NEAREST when an exact integer upscale was detected, so each captured pixel is
* replicated as an identical block instead of being interpolated.
*/
void bind_source_texture();

/**
* @brief Load ram data from the backing API or store.
*
Expand Down Expand Up @@ -764,6 +792,9 @@ namespace egl {
int offsetX; ///< Offset x.
int offsetY; ///< Offset y.

// Non-zero when the capture->stream blit uses point sampling instead of bilinear
int supersample {0}; ///< Integer capture-to-stream upscale factor, or 0 to keep bilinear filtering.

// Pointer to the texture to be converted to nv12
int loaded_texture; ///< Loaded texture.

Expand Down
1 change: 1 addition & 0 deletions src_assets/common/assets/web/config.html
Original file line number Diff line number Diff line change
Expand Up @@ -302,6 +302,7 @@ <h1>{{ $t('config.configuration') }}</h1>
"min_threads": 2,
"hevc_mode": 0,
"av1_mode": 0,
"chroma_supersample": "disabled",
"capture": "",
"encoder": "",
},
Expand Down
10 changes: 10 additions & 0 deletions src_assets/common/assets/web/configs/tabs/Advanced.vue
Original file line number Diff line number Diff line change
@@ -1,6 +1,7 @@
<script setup>
import { ref } from 'vue'
import PlatformLayout from '../../PlatformLayout.vue'
import Checkbox from '../../Checkbox.vue'

const props = defineProps([
'platform',
Expand Down Expand Up @@ -58,6 +59,15 @@ const config = ref(props.config)
<div class="form-text">{{ $t('config.av1_mode_desc') }}</div>
</div>

<!-- Chroma Supersample -->
<Checkbox class="mb-3"
v-if="platform === 'linux'"
id="chroma_supersample"
locale-prefix="config"
v-model="config.chroma_supersample"
default="false"
></Checkbox>

<!-- Capture -->
<div class="mb-3" v-if="platform !== 'macos'">
<label for="capture" class="form-label">{{ $t('config.capture') }}</label>
Expand Down
2 changes: 2 additions & 0 deletions src_assets/common/assets/web/public/assets/locale/en.json
Original file line number Diff line number Diff line change
Expand Up @@ -173,6 +173,8 @@
"channels": "Maximum Connected Clients",
"channels_desc_1": "Sunshine can allow a single streaming session to be shared with multiple clients simultaneously.",
"channels_desc_2": "Some hardware encoders may have limitations that reduce performance with multiple streams.",
"chroma_supersample": "Exact Chroma via Integer Supersampling",
"chroma_supersample_desc": "When the stream resolution is an exact, even multiple of the capture resolution (2x, 4x, ...), use nearest-neighbor upscaling instead of bilinear so 4:2:0 chroma subsampling reproduces the source exactly. This is a practical alternative to 4:4:4 encoding on GPUs without a 4:4:4 hardware encoder profile. Falls back to normal scaling automatically when the resolution ratio doesn't qualify. The client must play back fullscreen for the exactness to hold end to end.",
"coder_cabac": "cabac -- context adaptive binary arithmetic coding - higher quality",
"coder_cavlc": "cavlc -- context adaptive variable-length coding - faster decode",
"configuration": "Configuration",
Expand Down
52 changes: 52 additions & 0 deletions tests/unit/platform/linux/test_graphics.cpp
Original file line number Diff line number Diff line change
@@ -0,0 +1,52 @@
/**
* @file tests/unit/platform/linux/test_graphics.cpp
* @brief Test the chroma_supersample integer-factor detection.
*/
#if defined(__linux__)
#include "../../../tests_common.h"

#include <src/platform/linux/graphics.h>

// The exactness property chroma_supersample relies on only holds for an exact, even
// integer upscale in both axes - see src/platform/linux/graphics.cpp for the derivation.
// These cases mirror the values verified against a numpy simulation of the GL sampling
// rules during development: 2x/4x/6x/8x/10x/12x are bit-exact, 3x/5x/7x are not (and are
// measurably worse than the existing bilinear path), so supersample_factor() must reject
// odd factors rather than merely treat them as "less exact."
TEST(SupersampleFactorTest, AcceptsEvenIntegerFactors) {
EXPECT_EQ(egl::supersample_factor(1920, 1080, 3840, 2160), 2);
EXPECT_EQ(egl::supersample_factor(1920, 1080, 7680, 4320), 4);
EXPECT_EQ(egl::supersample_factor(480, 270, 2880, 1620), 6);
EXPECT_EQ(egl::supersample_factor(480, 270, 3840, 2160), 8);
EXPECT_EQ(egl::supersample_factor(480, 270, 4800, 2700), 10);
EXPECT_EQ(egl::supersample_factor(480, 270, 5760, 3240), 12);
}

TEST(SupersampleFactorTest, RejectsOddFactors) {
EXPECT_EQ(egl::supersample_factor(1920, 1080, 5760, 3240), 0); // 3x
EXPECT_EQ(egl::supersample_factor(480, 270, 2400, 1350), 0); // 5x
EXPECT_EQ(egl::supersample_factor(480, 270, 3360, 1890), 0); // 7x
}

TEST(SupersampleFactorTest, RejectsBelowMinimumFactor) {
// Same resolution (1x) and any downscale must not enable point sampling.
EXPECT_EQ(egl::supersample_factor(1920, 1080, 1920, 1080), 0);
EXPECT_EQ(egl::supersample_factor(1920, 1080, 1280, 720), 0);
}

TEST(SupersampleFactorTest, RejectsNonIntegerRatio) {
// 1.5x - not an integer multiple in either axis.
EXPECT_EQ(egl::supersample_factor(1920, 1080, 2880, 1620), 0);
}

TEST(SupersampleFactorTest, RejectsAnisotropicRatio) {
// Width scales 2x, height scales 3x - axes disagree, so no uniform block replication.
EXPECT_EQ(egl::supersample_factor(1920, 1080, 3840, 3240), 0);
}

TEST(SupersampleFactorTest, RejectsNonPositiveCaptureSize) {
EXPECT_EQ(egl::supersample_factor(0, 1080, 3840, 2160), 0);
EXPECT_EQ(egl::supersample_factor(1920, 0, 3840, 2160), 0);
EXPECT_EQ(egl::supersample_factor(-1920, 1080, 3840, 2160), 0);
}
#endif