When comparing stereo encodings, I want to know what happened to each eye. A single number for a packed frame can hide an uneven result: one view may survive compression better than the other, even though both contribute to the same viewing experience.
I put together a shell script to make variants of a side-by-side HEVC source and compare them with MV-HEVC versions. It uses FFmpeg for the ordinary HEVC path and Mike Swanson's Spatial command-line tool for the multiview path. The intended outputs are encoded files and left/right PSNR measurements at several target bitrates.
The useful subject here is the measurement process. There are no results that justify declaring one encoding path the winner.
Keep the reference unambiguous
The script assumes a 5760 by 2880 side-by-side input at 50 frames per second. Each eye is therefore a 2880 by 2880 image. It extracts those regions from the reference and from the encoded versions so that the left eye is compared with the left eye, and the right with the right.
This sounds obvious until a conversion changes the eye order, crop, pixel format or timing. A one-frame offset in a moving scene can overwhelm the differences caused by compression. A mismatched color conversion can also create error that belongs to the pipeline rather than the encoder.
Before trusting any quality metric, I would check frame counts, dimensions, eye assignment and representative decoded frames. Those checks are part of the comparison, not administrative work around it.
PSNR is based on mean squared pixel error. For a chosen signal range, a lower error produces a higher PSNR. It is useful because the calculation is explicit and reproducible. It is limited because equal pixel error does not always mean equal visible disturbance. Fine ringing around an edge and a small smooth change over a large region can affect perception differently.
Stereo introduces another layer. A metric calculated independently for each eye does not directly measure whether the pair is comfortable to fuse. It is a diagnostic, not a replacement for viewing the stereo result.
A row in a CSV is not automatically an average
There is a concrete limitation in the script that matters before anyone reuses its numbers. FFmpeg writes per-frame statistics to the requested stats file. The script extracts psnr_avg and then takes the final matching line with tail -n 1. That selects the last frame's value, not an aggregate over the sequence.
The script then averages the two eye values in decibels. This is another choice that needs care. PSNR is logarithmic. Averaging two dB values is not generally the same as combining the underlying squared errors and then calculating PSNR from the combined error.
For a sequence-level comparison, I would accumulate error over the intended frames and samples, document the weighting, and calculate the final metric from that result. I would also retain per-eye and per-frame information. A sequence average can hide a brief but severe failure that is easy to notice during playback.
This is a useful example of why a successful command is only one part of a benchmark. The encoder can finish, the decoder can finish and the CSV can contain plausible numbers while the aggregation answers a different question from the column heading.
Compare what was actually produced
The bitrate loop requests targets from 2 to 16 Mbps. A target is not the same as the achieved bitrate. Encoders may distribute bits differently over time, and container overhead also exists. A fair comparison should report the actual size and duration of the output, alongside the settings used to create it.
The two paths also use different encoding tools. That means the experiment compares practical pipelines under specified settings, not an isolated mathematical property of the codecs. Presets, rate control and implementation details can all influence the result.
Spatial metadata deserves its own check. The script supplies values for camera distance, field of view and horizontal adjustment. Those values should match the material being encoded. Copying them into another capture without understanding the geometry would create a new problem unrelated to compression quality.
I like having a script because it makes these assumptions visible. The next useful step is to make the aggregation and validation as explicit as the encoding loop, then run a reproducible comparison on identified source clips. Until that exists, the script is a starting point for measurement. Its most valuable output is a clear list of questions to resolve before a graph earns anyone's trust.