1. To remove MJ just uninstall MobileJoomla component.
2. MJ shows mobile pages in the following cases only:
a) Browser's UserAgent string is determined as mobile devices (based on Mobile-Simple or Mobile-TeraWURFL plugin). This case may be tested at
www.mobilejoomla.com/uatest.php
b) Mobile view was selected with corresponding link in "Select Markup" module. This case may be tested by clearing of cookies.
c) You visit mobile subdomain. There is no to test as it's clear from address bar.
The fact that mobile template is shows on desktop pc most likely means that there is some problems with caching. First of all I'd check:
a) System-Cache plugin is either disabled or runned after "Mobile Joomla Plugin".
b) If you use frontend+backend servers the check that frontend server is not able to mix cached data for different devices.