I thought the answer was "no", but I just tried it. Sure enough, when I plug two different sources into the left and right inputs, and connect just one output (either one) to my amp, I get both input signals in the output.
Further, when I plug a cable into the second output, I hear only one input. (There is some crosstalk, but not much.)