x86/mm: fix a reference counting error in MMU_MACHPHYS_UPDATE

Any domain which can pass the XSM check against a translated guest can cause a
page reference to be leaked.

While shuffling the order of checks, drop the quite-pointless MEM_LOG().  This
brings the check in line with similar checks in the vicinity.

Discovered while reviewing the XSA-109/110 followup series.

This is XSA-113.

Signed-off-by: Andrew Cooper <andrew.cooper3@citrix.com>
Reviewed-by: Jan Beulich <jbeulich@suse.com>
Reviewed-by: Tim Deegan <tim@xen.org>

--- a/xen/arch/x86/mm.c
+++ b/xen/arch/x86/mm.c
@@ -3619,6 +3619,12 @@ long do_mmu_update(
 
         case MMU_MACHPHYS_UPDATE:
 
+            if ( unlikely(paging_mode_translate(pg_owner)) )
+            {
+                rc = -EINVAL;
+                break;
+            }
+
             mfn = req.ptr >> PAGE_SHIFT;
             gpfn = req.val;
 
@@ -3638,13 +3644,6 @@ long do_mmu_update(
                 break;
             }
 
-            if ( unlikely(paging_mode_translate(pg_owner)) )
-            {
-                MEM_LOG("Mach-phys update on auto-translate guest");
-                rc = -EINVAL;
-                break;
-            }
-
             set_gpfn_from_mfn(mfn, gpfn);
 
             paging_mark_dirty(pg_owner, mfn);