“The person viewing your work has no idea what the scene really looked like, nor do they care…”
– Mike Svob
It certainly is true that no one who was not there with you really knows what you saw and, even if they were there, we all know that each of us sees differently. Given the same put, most of us would come up with at least a slightly different output.
But do our viewers care what the scene looked like? Often we want them to. We try to depict as faithfully as possible what we saw so others can share it too. We want them to share our experience of it. There is a genre of art the might accomplish this, work that is representational in nature that invites the viewer to stand next to us and see what we saw.
I hope my viewers don’t really care what the scene looked like. I hope they care what it caused me to think or feel. That’s what I want to share, not the scene itself.
I hope my viewers want to take a different journey with me, not back to the place I was, but to the place I went afterwards. To the place populated by my reactions, interpretations and impressions of the original scene.
Caring about what the scene really looked like means caring less about the work of art that it inspired. Focusing on the scene can be a distraction from the art. It is the art which is my offering, the thing I wish to share with you.
Bob Cornelis' Art Blog 
I’ve rarely read a more perfect description of what an artist wants from the viewer. You seem to be sitting in a place of such clarity about the process of art.