It's because of detecting of mobile mode (desktop, smartphone, iphone, or tablet) is based on User-Agent header. I don't know other way to prevent incorrect serving by proxies.
In the case of Varnish it would be possible to write vcl_recv subroutine to detect device type, but it is specific case, so I think that Vary: User-Agent should be enabled by default. And anyway, it is not easy to transform mobile detetcting plugins to Varnish subroutine (if possible).