|
![](https://seccdn.libravatar.org/avatar/f1a0ddfc363b14e167bf8548e95f340032d1964ae5b6ea4f235e351fd948008e?s=16&d=retro) |
161d5c0 |
From: Andrew Cooper <andrew.cooper3@citrix.com>
|
|
![](https://seccdn.libravatar.org/avatar/f1a0ddfc363b14e167bf8548e95f340032d1964ae5b6ea4f235e351fd948008e?s=16&d=retro) |
161d5c0 |
Subject: x86/pv: Fix ABAC cmpxchg() race in _get_page_type()
|
|
![](https://seccdn.libravatar.org/avatar/f1a0ddfc363b14e167bf8548e95f340032d1964ae5b6ea4f235e351fd948008e?s=16&d=retro) |
161d5c0 |
|
|
![](https://seccdn.libravatar.org/avatar/f1a0ddfc363b14e167bf8548e95f340032d1964ae5b6ea4f235e351fd948008e?s=16&d=retro) |
161d5c0 |
_get_page_type() suffers from a race condition where it incorrectly assumes
|
|
![](https://seccdn.libravatar.org/avatar/f1a0ddfc363b14e167bf8548e95f340032d1964ae5b6ea4f235e351fd948008e?s=16&d=retro) |
161d5c0 |
that because 'x' was read and a subsequent a cmpxchg() succeeds, the type
|
|
![](https://seccdn.libravatar.org/avatar/f1a0ddfc363b14e167bf8548e95f340032d1964ae5b6ea4f235e351fd948008e?s=16&d=retro) |
161d5c0 |
cannot have changed in-between. Consider:
|
|
![](https://seccdn.libravatar.org/avatar/f1a0ddfc363b14e167bf8548e95f340032d1964ae5b6ea4f235e351fd948008e?s=16&d=retro) |
161d5c0 |
|
|
![](https://seccdn.libravatar.org/avatar/f1a0ddfc363b14e167bf8548e95f340032d1964ae5b6ea4f235e351fd948008e?s=16&d=retro) |
161d5c0 |
CPU A:
|
|
![](https://seccdn.libravatar.org/avatar/f1a0ddfc363b14e167bf8548e95f340032d1964ae5b6ea4f235e351fd948008e?s=16&d=retro) |
161d5c0 |
1. Creates an L2e referencing pg
|
|
![](https://seccdn.libravatar.org/avatar/f1a0ddfc363b14e167bf8548e95f340032d1964ae5b6ea4f235e351fd948008e?s=16&d=retro) |
161d5c0 |
`-> _get_page_type(pg, PGT_l1_page_table), sees count 0, type PGT_writable_page
|
|
![](https://seccdn.libravatar.org/avatar/f1a0ddfc363b14e167bf8548e95f340032d1964ae5b6ea4f235e351fd948008e?s=16&d=retro) |
161d5c0 |
2. Issues flush_tlb_mask()
|
|
![](https://seccdn.libravatar.org/avatar/f1a0ddfc363b14e167bf8548e95f340032d1964ae5b6ea4f235e351fd948008e?s=16&d=retro) |
161d5c0 |
CPU B:
|
|
![](https://seccdn.libravatar.org/avatar/f1a0ddfc363b14e167bf8548e95f340032d1964ae5b6ea4f235e351fd948008e?s=16&d=retro) |
161d5c0 |
3. Creates a writeable mapping of pg
|
|
![](https://seccdn.libravatar.org/avatar/f1a0ddfc363b14e167bf8548e95f340032d1964ae5b6ea4f235e351fd948008e?s=16&d=retro) |
161d5c0 |
`-> _get_page_type(pg, PGT_writable_page), count increases to 1
|
|
![](https://seccdn.libravatar.org/avatar/f1a0ddfc363b14e167bf8548e95f340032d1964ae5b6ea4f235e351fd948008e?s=16&d=retro) |
161d5c0 |
4. Writes into new mapping, creating a TLB entry for pg
|
|
![](https://seccdn.libravatar.org/avatar/f1a0ddfc363b14e167bf8548e95f340032d1964ae5b6ea4f235e351fd948008e?s=16&d=retro) |
161d5c0 |
5. Removes the writeable mapping of pg
|
|
![](https://seccdn.libravatar.org/avatar/f1a0ddfc363b14e167bf8548e95f340032d1964ae5b6ea4f235e351fd948008e?s=16&d=retro) |
161d5c0 |
`-> _put_page_type(pg), count goes back down to 0
|
|
![](https://seccdn.libravatar.org/avatar/f1a0ddfc363b14e167bf8548e95f340032d1964ae5b6ea4f235e351fd948008e?s=16&d=retro) |
161d5c0 |
CPU A:
|
|
![](https://seccdn.libravatar.org/avatar/f1a0ddfc363b14e167bf8548e95f340032d1964ae5b6ea4f235e351fd948008e?s=16&d=retro) |
161d5c0 |
7. Issues cmpxchg(), setting count 1, type PGT_l1_page_table
|
|
![](https://seccdn.libravatar.org/avatar/f1a0ddfc363b14e167bf8548e95f340032d1964ae5b6ea4f235e351fd948008e?s=16&d=retro) |
161d5c0 |
|
|
![](https://seccdn.libravatar.org/avatar/f1a0ddfc363b14e167bf8548e95f340032d1964ae5b6ea4f235e351fd948008e?s=16&d=retro) |
161d5c0 |
CPU B now has a writeable mapping to pg, which Xen believes is a pagetable and
|
|
![](https://seccdn.libravatar.org/avatar/f1a0ddfc363b14e167bf8548e95f340032d1964ae5b6ea4f235e351fd948008e?s=16&d=retro) |
161d5c0 |
suitably protected (i.e. read-only). The TLB flush in step 2 must be deferred
|
|
![](https://seccdn.libravatar.org/avatar/f1a0ddfc363b14e167bf8548e95f340032d1964ae5b6ea4f235e351fd948008e?s=16&d=retro) |
161d5c0 |
until after the guest is prohibited from creating new writeable mappings,
|
|
![](https://seccdn.libravatar.org/avatar/f1a0ddfc363b14e167bf8548e95f340032d1964ae5b6ea4f235e351fd948008e?s=16&d=retro) |
161d5c0 |
which is after step 7.
|
|
![](https://seccdn.libravatar.org/avatar/f1a0ddfc363b14e167bf8548e95f340032d1964ae5b6ea4f235e351fd948008e?s=16&d=retro) |
161d5c0 |
|
|
![](https://seccdn.libravatar.org/avatar/f1a0ddfc363b14e167bf8548e95f340032d1964ae5b6ea4f235e351fd948008e?s=16&d=retro) |
161d5c0 |
Defer all safety actions until after the cmpxchg() has successfully taken the
|
|
![](https://seccdn.libravatar.org/avatar/f1a0ddfc363b14e167bf8548e95f340032d1964ae5b6ea4f235e351fd948008e?s=16&d=retro) |
161d5c0 |
intended typeref, because that is what prevents concurrent users from using
|
|
![](https://seccdn.libravatar.org/avatar/f1a0ddfc363b14e167bf8548e95f340032d1964ae5b6ea4f235e351fd948008e?s=16&d=retro) |
161d5c0 |
the old type.
|
|
![](https://seccdn.libravatar.org/avatar/f1a0ddfc363b14e167bf8548e95f340032d1964ae5b6ea4f235e351fd948008e?s=16&d=retro) |
161d5c0 |
|
|
![](https://seccdn.libravatar.org/avatar/f1a0ddfc363b14e167bf8548e95f340032d1964ae5b6ea4f235e351fd948008e?s=16&d=retro) |
161d5c0 |
Also remove the early validation for writeable and shared pages. This removes
|
|
![](https://seccdn.libravatar.org/avatar/f1a0ddfc363b14e167bf8548e95f340032d1964ae5b6ea4f235e351fd948008e?s=16&d=retro) |
161d5c0 |
race conditions where one half of a parallel mapping attempt can return
|
|
![](https://seccdn.libravatar.org/avatar/f1a0ddfc363b14e167bf8548e95f340032d1964ae5b6ea4f235e351fd948008e?s=16&d=retro) |
161d5c0 |
successfully before:
|
|
![](https://seccdn.libravatar.org/avatar/f1a0ddfc363b14e167bf8548e95f340032d1964ae5b6ea4f235e351fd948008e?s=16&d=retro) |
161d5c0 |
* The IOMMU pagetables are in sync with the new page type
|
|
![](https://seccdn.libravatar.org/avatar/f1a0ddfc363b14e167bf8548e95f340032d1964ae5b6ea4f235e351fd948008e?s=16&d=retro) |
161d5c0 |
* Writeable mappings to shared pages have been torn down
|
|
![](https://seccdn.libravatar.org/avatar/f1a0ddfc363b14e167bf8548e95f340032d1964ae5b6ea4f235e351fd948008e?s=16&d=retro) |
161d5c0 |
|
|
![](https://seccdn.libravatar.org/avatar/f1a0ddfc363b14e167bf8548e95f340032d1964ae5b6ea4f235e351fd948008e?s=16&d=retro) |
161d5c0 |
This is part of XSA-401 / CVE-2022-26362.
|
|
![](https://seccdn.libravatar.org/avatar/f1a0ddfc363b14e167bf8548e95f340032d1964ae5b6ea4f235e351fd948008e?s=16&d=retro) |
161d5c0 |
|
|
![](https://seccdn.libravatar.org/avatar/f1a0ddfc363b14e167bf8548e95f340032d1964ae5b6ea4f235e351fd948008e?s=16&d=retro) |
161d5c0 |
Reported-by: Jann Horn <jannh@google.com>
|
|
![](https://seccdn.libravatar.org/avatar/f1a0ddfc363b14e167bf8548e95f340032d1964ae5b6ea4f235e351fd948008e?s=16&d=retro) |
161d5c0 |
Signed-off-by: Andrew Cooper <andrew.cooper3@citrix.com>
|
|
![](https://seccdn.libravatar.org/avatar/f1a0ddfc363b14e167bf8548e95f340032d1964ae5b6ea4f235e351fd948008e?s=16&d=retro) |
161d5c0 |
Reviewed-by: Jan Beulich <jbeulich@suse.com>
|
|
![](https://seccdn.libravatar.org/avatar/f1a0ddfc363b14e167bf8548e95f340032d1964ae5b6ea4f235e351fd948008e?s=16&d=retro) |
161d5c0 |
Reviewed-by: George Dunlap <george.dunlap@citrix.com>
|
|
![](https://seccdn.libravatar.org/avatar/f1a0ddfc363b14e167bf8548e95f340032d1964ae5b6ea4f235e351fd948008e?s=16&d=retro) |
161d5c0 |
|
|
![](https://seccdn.libravatar.org/avatar/f1a0ddfc363b14e167bf8548e95f340032d1964ae5b6ea4f235e351fd948008e?s=16&d=retro) |
161d5c0 |
diff --git a/xen/arch/x86/mm.c b/xen/arch/x86/mm.c
|
|
![](https://seccdn.libravatar.org/avatar/f1a0ddfc363b14e167bf8548e95f340032d1964ae5b6ea4f235e351fd948008e?s=16&d=retro) |
161d5c0 |
index ddd32f88c798..1693b580b152 100644
|
|
![](https://seccdn.libravatar.org/avatar/f1a0ddfc363b14e167bf8548e95f340032d1964ae5b6ea4f235e351fd948008e?s=16&d=retro) |
161d5c0 |
--- a/xen/arch/x86/mm.c
|
|
![](https://seccdn.libravatar.org/avatar/f1a0ddfc363b14e167bf8548e95f340032d1964ae5b6ea4f235e351fd948008e?s=16&d=retro) |
161d5c0 |
+++ b/xen/arch/x86/mm.c
|
|
![](https://seccdn.libravatar.org/avatar/f1a0ddfc363b14e167bf8548e95f340032d1964ae5b6ea4f235e351fd948008e?s=16&d=retro) |
161d5c0 |
@@ -2962,56 +2962,12 @@ static int _get_page_type(struct page_info *page, unsigned long type,
|
|
![](https://seccdn.libravatar.org/avatar/f1a0ddfc363b14e167bf8548e95f340032d1964ae5b6ea4f235e351fd948008e?s=16&d=retro) |
161d5c0 |
* Type changes are permitted when the typeref is 0. If the type
|
|
![](https://seccdn.libravatar.org/avatar/f1a0ddfc363b14e167bf8548e95f340032d1964ae5b6ea4f235e351fd948008e?s=16&d=retro) |
161d5c0 |
* actually changes, the page needs re-validating.
|
|
![](https://seccdn.libravatar.org/avatar/f1a0ddfc363b14e167bf8548e95f340032d1964ae5b6ea4f235e351fd948008e?s=16&d=retro) |
161d5c0 |
*/
|
|
![](https://seccdn.libravatar.org/avatar/f1a0ddfc363b14e167bf8548e95f340032d1964ae5b6ea4f235e351fd948008e?s=16&d=retro) |
161d5c0 |
- struct domain *d = page_get_owner(page);
|
|
![](https://seccdn.libravatar.org/avatar/f1a0ddfc363b14e167bf8548e95f340032d1964ae5b6ea4f235e351fd948008e?s=16&d=retro) |
161d5c0 |
-
|
|
![](https://seccdn.libravatar.org/avatar/f1a0ddfc363b14e167bf8548e95f340032d1964ae5b6ea4f235e351fd948008e?s=16&d=retro) |
161d5c0 |
- if ( d && shadow_mode_enabled(d) )
|
|
![](https://seccdn.libravatar.org/avatar/f1a0ddfc363b14e167bf8548e95f340032d1964ae5b6ea4f235e351fd948008e?s=16&d=retro) |
161d5c0 |
- shadow_prepare_page_type_change(d, page, type);
|
|
![](https://seccdn.libravatar.org/avatar/f1a0ddfc363b14e167bf8548e95f340032d1964ae5b6ea4f235e351fd948008e?s=16&d=retro) |
161d5c0 |
|
|
![](https://seccdn.libravatar.org/avatar/f1a0ddfc363b14e167bf8548e95f340032d1964ae5b6ea4f235e351fd948008e?s=16&d=retro) |
161d5c0 |
ASSERT(!(x & PGT_pae_xen_l2));
|
|
![](https://seccdn.libravatar.org/avatar/f1a0ddfc363b14e167bf8548e95f340032d1964ae5b6ea4f235e351fd948008e?s=16&d=retro) |
161d5c0 |
if ( (x & PGT_type_mask) != type )
|
|
![](https://seccdn.libravatar.org/avatar/f1a0ddfc363b14e167bf8548e95f340032d1964ae5b6ea4f235e351fd948008e?s=16&d=retro) |
161d5c0 |
{
|
|
![](https://seccdn.libravatar.org/avatar/f1a0ddfc363b14e167bf8548e95f340032d1964ae5b6ea4f235e351fd948008e?s=16&d=retro) |
161d5c0 |
- /*
|
|
![](https://seccdn.libravatar.org/avatar/f1a0ddfc363b14e167bf8548e95f340032d1964ae5b6ea4f235e351fd948008e?s=16&d=retro) |
161d5c0 |
- * On type change we check to flush stale TLB entries. It is
|
|
![](https://seccdn.libravatar.org/avatar/f1a0ddfc363b14e167bf8548e95f340032d1964ae5b6ea4f235e351fd948008e?s=16&d=retro) |
161d5c0 |
- * vital that no other CPUs are left with writeable mappings
|
|
![](https://seccdn.libravatar.org/avatar/f1a0ddfc363b14e167bf8548e95f340032d1964ae5b6ea4f235e351fd948008e?s=16&d=retro) |
161d5c0 |
- * to a frame which is intending to become pgtable/segdesc.
|
|
![](https://seccdn.libravatar.org/avatar/f1a0ddfc363b14e167bf8548e95f340032d1964ae5b6ea4f235e351fd948008e?s=16&d=retro) |
161d5c0 |
- */
|
|
![](https://seccdn.libravatar.org/avatar/f1a0ddfc363b14e167bf8548e95f340032d1964ae5b6ea4f235e351fd948008e?s=16&d=retro) |
161d5c0 |
- cpumask_t *mask = this_cpu(scratch_cpumask);
|
|
![](https://seccdn.libravatar.org/avatar/f1a0ddfc363b14e167bf8548e95f340032d1964ae5b6ea4f235e351fd948008e?s=16&d=retro) |
161d5c0 |
-
|
|
![](https://seccdn.libravatar.org/avatar/f1a0ddfc363b14e167bf8548e95f340032d1964ae5b6ea4f235e351fd948008e?s=16&d=retro) |
161d5c0 |
- BUG_ON(in_irq());
|
|
![](https://seccdn.libravatar.org/avatar/f1a0ddfc363b14e167bf8548e95f340032d1964ae5b6ea4f235e351fd948008e?s=16&d=retro) |
161d5c0 |
- cpumask_copy(mask, d->dirty_cpumask);
|
|
![](https://seccdn.libravatar.org/avatar/f1a0ddfc363b14e167bf8548e95f340032d1964ae5b6ea4f235e351fd948008e?s=16&d=retro) |
161d5c0 |
-
|
|
![](https://seccdn.libravatar.org/avatar/f1a0ddfc363b14e167bf8548e95f340032d1964ae5b6ea4f235e351fd948008e?s=16&d=retro) |
161d5c0 |
- /* Don't flush if the timestamp is old enough */
|
|
![](https://seccdn.libravatar.org/avatar/f1a0ddfc363b14e167bf8548e95f340032d1964ae5b6ea4f235e351fd948008e?s=16&d=retro) |
161d5c0 |
- tlbflush_filter(mask, page->tlbflush_timestamp);
|
|
![](https://seccdn.libravatar.org/avatar/f1a0ddfc363b14e167bf8548e95f340032d1964ae5b6ea4f235e351fd948008e?s=16&d=retro) |
161d5c0 |
-
|
|
![](https://seccdn.libravatar.org/avatar/f1a0ddfc363b14e167bf8548e95f340032d1964ae5b6ea4f235e351fd948008e?s=16&d=retro) |
161d5c0 |
- if ( unlikely(!cpumask_empty(mask)) &&
|
|
![](https://seccdn.libravatar.org/avatar/f1a0ddfc363b14e167bf8548e95f340032d1964ae5b6ea4f235e351fd948008e?s=16&d=retro) |
161d5c0 |
- /* Shadow mode: track only writable pages. */
|
|
![](https://seccdn.libravatar.org/avatar/f1a0ddfc363b14e167bf8548e95f340032d1964ae5b6ea4f235e351fd948008e?s=16&d=retro) |
161d5c0 |
- (!shadow_mode_enabled(d) ||
|
|
![](https://seccdn.libravatar.org/avatar/f1a0ddfc363b14e167bf8548e95f340032d1964ae5b6ea4f235e351fd948008e?s=16&d=retro) |
161d5c0 |
- ((nx & PGT_type_mask) == PGT_writable_page)) )
|
|
![](https://seccdn.libravatar.org/avatar/f1a0ddfc363b14e167bf8548e95f340032d1964ae5b6ea4f235e351fd948008e?s=16&d=retro) |
161d5c0 |
- {
|
|
![](https://seccdn.libravatar.org/avatar/f1a0ddfc363b14e167bf8548e95f340032d1964ae5b6ea4f235e351fd948008e?s=16&d=retro) |
161d5c0 |
- perfc_incr(need_flush_tlb_flush);
|
|
![](https://seccdn.libravatar.org/avatar/f1a0ddfc363b14e167bf8548e95f340032d1964ae5b6ea4f235e351fd948008e?s=16&d=retro) |
161d5c0 |
- /*
|
|
![](https://seccdn.libravatar.org/avatar/f1a0ddfc363b14e167bf8548e95f340032d1964ae5b6ea4f235e351fd948008e?s=16&d=retro) |
161d5c0 |
- * If page was a page table make sure the flush is
|
|
![](https://seccdn.libravatar.org/avatar/f1a0ddfc363b14e167bf8548e95f340032d1964ae5b6ea4f235e351fd948008e?s=16&d=retro) |
161d5c0 |
- * performed using an IPI in order to avoid changing the
|
|
![](https://seccdn.libravatar.org/avatar/f1a0ddfc363b14e167bf8548e95f340032d1964ae5b6ea4f235e351fd948008e?s=16&d=retro) |
161d5c0 |
- * type of a page table page under the feet of
|
|
![](https://seccdn.libravatar.org/avatar/f1a0ddfc363b14e167bf8548e95f340032d1964ae5b6ea4f235e351fd948008e?s=16&d=retro) |
161d5c0 |
- * spurious_page_fault().
|
|
![](https://seccdn.libravatar.org/avatar/f1a0ddfc363b14e167bf8548e95f340032d1964ae5b6ea4f235e351fd948008e?s=16&d=retro) |
161d5c0 |
- */
|
|
![](https://seccdn.libravatar.org/avatar/f1a0ddfc363b14e167bf8548e95f340032d1964ae5b6ea4f235e351fd948008e?s=16&d=retro) |
161d5c0 |
- flush_mask(mask,
|
|
![](https://seccdn.libravatar.org/avatar/f1a0ddfc363b14e167bf8548e95f340032d1964ae5b6ea4f235e351fd948008e?s=16&d=retro) |
161d5c0 |
- (x & PGT_type_mask) &&
|
|
![](https://seccdn.libravatar.org/avatar/f1a0ddfc363b14e167bf8548e95f340032d1964ae5b6ea4f235e351fd948008e?s=16&d=retro) |
161d5c0 |
- (x & PGT_type_mask) <= PGT_root_page_table
|
|
![](https://seccdn.libravatar.org/avatar/f1a0ddfc363b14e167bf8548e95f340032d1964ae5b6ea4f235e351fd948008e?s=16&d=retro) |
161d5c0 |
- ? FLUSH_TLB | FLUSH_FORCE_IPI
|
|
![](https://seccdn.libravatar.org/avatar/f1a0ddfc363b14e167bf8548e95f340032d1964ae5b6ea4f235e351fd948008e?s=16&d=retro) |
161d5c0 |
- : FLUSH_TLB);
|
|
![](https://seccdn.libravatar.org/avatar/f1a0ddfc363b14e167bf8548e95f340032d1964ae5b6ea4f235e351fd948008e?s=16&d=retro) |
161d5c0 |
- }
|
|
![](https://seccdn.libravatar.org/avatar/f1a0ddfc363b14e167bf8548e95f340032d1964ae5b6ea4f235e351fd948008e?s=16&d=retro) |
161d5c0 |
-
|
|
![](https://seccdn.libravatar.org/avatar/f1a0ddfc363b14e167bf8548e95f340032d1964ae5b6ea4f235e351fd948008e?s=16&d=retro) |
161d5c0 |
- /* We lose existing type and validity. */
|
|
![](https://seccdn.libravatar.org/avatar/f1a0ddfc363b14e167bf8548e95f340032d1964ae5b6ea4f235e351fd948008e?s=16&d=retro) |
161d5c0 |
nx &= ~(PGT_type_mask | PGT_validated);
|
|
![](https://seccdn.libravatar.org/avatar/f1a0ddfc363b14e167bf8548e95f340032d1964ae5b6ea4f235e351fd948008e?s=16&d=retro) |
161d5c0 |
nx |= type;
|
|
![](https://seccdn.libravatar.org/avatar/f1a0ddfc363b14e167bf8548e95f340032d1964ae5b6ea4f235e351fd948008e?s=16&d=retro) |
161d5c0 |
-
|
|
![](https://seccdn.libravatar.org/avatar/f1a0ddfc363b14e167bf8548e95f340032d1964ae5b6ea4f235e351fd948008e?s=16&d=retro) |
161d5c0 |
- /*
|
|
![](https://seccdn.libravatar.org/avatar/f1a0ddfc363b14e167bf8548e95f340032d1964ae5b6ea4f235e351fd948008e?s=16&d=retro) |
161d5c0 |
- * No special validation needed for writable pages.
|
|
![](https://seccdn.libravatar.org/avatar/f1a0ddfc363b14e167bf8548e95f340032d1964ae5b6ea4f235e351fd948008e?s=16&d=retro) |
161d5c0 |
- * Page tables and GDT/LDT need to be scanned for validity.
|
|
![](https://seccdn.libravatar.org/avatar/f1a0ddfc363b14e167bf8548e95f340032d1964ae5b6ea4f235e351fd948008e?s=16&d=retro) |
161d5c0 |
- */
|
|
![](https://seccdn.libravatar.org/avatar/f1a0ddfc363b14e167bf8548e95f340032d1964ae5b6ea4f235e351fd948008e?s=16&d=retro) |
161d5c0 |
- if ( type == PGT_writable_page || type == PGT_shared_page )
|
|
![](https://seccdn.libravatar.org/avatar/f1a0ddfc363b14e167bf8548e95f340032d1964ae5b6ea4f235e351fd948008e?s=16&d=retro) |
161d5c0 |
- nx |= PGT_validated;
|
|
![](https://seccdn.libravatar.org/avatar/f1a0ddfc363b14e167bf8548e95f340032d1964ae5b6ea4f235e351fd948008e?s=16&d=retro) |
161d5c0 |
}
|
|
![](https://seccdn.libravatar.org/avatar/f1a0ddfc363b14e167bf8548e95f340032d1964ae5b6ea4f235e351fd948008e?s=16&d=retro) |
161d5c0 |
}
|
|
![](https://seccdn.libravatar.org/avatar/f1a0ddfc363b14e167bf8548e95f340032d1964ae5b6ea4f235e351fd948008e?s=16&d=retro) |
161d5c0 |
else if ( unlikely((x & (PGT_type_mask|PGT_pae_xen_l2)) != type) )
|
|
![](https://seccdn.libravatar.org/avatar/f1a0ddfc363b14e167bf8548e95f340032d1964ae5b6ea4f235e351fd948008e?s=16&d=retro) |
161d5c0 |
@@ -3092,6 +3048,56 @@ static int _get_page_type(struct page_info *page, unsigned long type,
|
|
![](https://seccdn.libravatar.org/avatar/f1a0ddfc363b14e167bf8548e95f340032d1964ae5b6ea4f235e351fd948008e?s=16&d=retro) |
161d5c0 |
return -EINTR;
|
|
![](https://seccdn.libravatar.org/avatar/f1a0ddfc363b14e167bf8548e95f340032d1964ae5b6ea4f235e351fd948008e?s=16&d=retro) |
161d5c0 |
}
|
|
![](https://seccdn.libravatar.org/avatar/f1a0ddfc363b14e167bf8548e95f340032d1964ae5b6ea4f235e351fd948008e?s=16&d=retro) |
161d5c0 |
|
|
![](https://seccdn.libravatar.org/avatar/f1a0ddfc363b14e167bf8548e95f340032d1964ae5b6ea4f235e351fd948008e?s=16&d=retro) |
161d5c0 |
+ /*
|
|
![](https://seccdn.libravatar.org/avatar/f1a0ddfc363b14e167bf8548e95f340032d1964ae5b6ea4f235e351fd948008e?s=16&d=retro) |
161d5c0 |
+ * One typeref has been taken and is now globally visible.
|
|
![](https://seccdn.libravatar.org/avatar/f1a0ddfc363b14e167bf8548e95f340032d1964ae5b6ea4f235e351fd948008e?s=16&d=retro) |
161d5c0 |
+ *
|
|
![](https://seccdn.libravatar.org/avatar/f1a0ddfc363b14e167bf8548e95f340032d1964ae5b6ea4f235e351fd948008e?s=16&d=retro) |
161d5c0 |
+ * The page is either in the "validate locked" state (PGT_[type] | 1) or
|
|
![](https://seccdn.libravatar.org/avatar/f1a0ddfc363b14e167bf8548e95f340032d1964ae5b6ea4f235e351fd948008e?s=16&d=retro) |
161d5c0 |
+ * fully validated (PGT_[type] | PGT_validated | >0).
|
|
![](https://seccdn.libravatar.org/avatar/f1a0ddfc363b14e167bf8548e95f340032d1964ae5b6ea4f235e351fd948008e?s=16&d=retro) |
161d5c0 |
+ */
|
|
![](https://seccdn.libravatar.org/avatar/f1a0ddfc363b14e167bf8548e95f340032d1964ae5b6ea4f235e351fd948008e?s=16&d=retro) |
161d5c0 |
+
|
|
![](https://seccdn.libravatar.org/avatar/f1a0ddfc363b14e167bf8548e95f340032d1964ae5b6ea4f235e351fd948008e?s=16&d=retro) |
161d5c0 |
+ if ( unlikely((x & PGT_count_mask) == 0) )
|
|
![](https://seccdn.libravatar.org/avatar/f1a0ddfc363b14e167bf8548e95f340032d1964ae5b6ea4f235e351fd948008e?s=16&d=retro) |
161d5c0 |
+ {
|
|
![](https://seccdn.libravatar.org/avatar/f1a0ddfc363b14e167bf8548e95f340032d1964ae5b6ea4f235e351fd948008e?s=16&d=retro) |
161d5c0 |
+ struct domain *d = page_get_owner(page);
|
|
![](https://seccdn.libravatar.org/avatar/f1a0ddfc363b14e167bf8548e95f340032d1964ae5b6ea4f235e351fd948008e?s=16&d=retro) |
161d5c0 |
+
|
|
![](https://seccdn.libravatar.org/avatar/f1a0ddfc363b14e167bf8548e95f340032d1964ae5b6ea4f235e351fd948008e?s=16&d=retro) |
161d5c0 |
+ if ( d && shadow_mode_enabled(d) )
|
|
![](https://seccdn.libravatar.org/avatar/f1a0ddfc363b14e167bf8548e95f340032d1964ae5b6ea4f235e351fd948008e?s=16&d=retro) |
161d5c0 |
+ shadow_prepare_page_type_change(d, page, type);
|
|
![](https://seccdn.libravatar.org/avatar/f1a0ddfc363b14e167bf8548e95f340032d1964ae5b6ea4f235e351fd948008e?s=16&d=retro) |
161d5c0 |
+
|
|
![](https://seccdn.libravatar.org/avatar/f1a0ddfc363b14e167bf8548e95f340032d1964ae5b6ea4f235e351fd948008e?s=16&d=retro) |
161d5c0 |
+ if ( (x & PGT_type_mask) != type )
|
|
![](https://seccdn.libravatar.org/avatar/f1a0ddfc363b14e167bf8548e95f340032d1964ae5b6ea4f235e351fd948008e?s=16&d=retro) |
161d5c0 |
+ {
|
|
![](https://seccdn.libravatar.org/avatar/f1a0ddfc363b14e167bf8548e95f340032d1964ae5b6ea4f235e351fd948008e?s=16&d=retro) |
161d5c0 |
+ /*
|
|
![](https://seccdn.libravatar.org/avatar/f1a0ddfc363b14e167bf8548e95f340032d1964ae5b6ea4f235e351fd948008e?s=16&d=retro) |
161d5c0 |
+ * On type change we check to flush stale TLB entries. It is
|
|
![](https://seccdn.libravatar.org/avatar/f1a0ddfc363b14e167bf8548e95f340032d1964ae5b6ea4f235e351fd948008e?s=16&d=retro) |
161d5c0 |
+ * vital that no other CPUs are left with writeable mappings
|
|
![](https://seccdn.libravatar.org/avatar/f1a0ddfc363b14e167bf8548e95f340032d1964ae5b6ea4f235e351fd948008e?s=16&d=retro) |
161d5c0 |
+ * to a frame which is intending to become pgtable/segdesc.
|
|
![](https://seccdn.libravatar.org/avatar/f1a0ddfc363b14e167bf8548e95f340032d1964ae5b6ea4f235e351fd948008e?s=16&d=retro) |
161d5c0 |
+ */
|
|
![](https://seccdn.libravatar.org/avatar/f1a0ddfc363b14e167bf8548e95f340032d1964ae5b6ea4f235e351fd948008e?s=16&d=retro) |
161d5c0 |
+ cpumask_t *mask = this_cpu(scratch_cpumask);
|
|
![](https://seccdn.libravatar.org/avatar/f1a0ddfc363b14e167bf8548e95f340032d1964ae5b6ea4f235e351fd948008e?s=16&d=retro) |
161d5c0 |
+
|
|
![](https://seccdn.libravatar.org/avatar/f1a0ddfc363b14e167bf8548e95f340032d1964ae5b6ea4f235e351fd948008e?s=16&d=retro) |
161d5c0 |
+ BUG_ON(in_irq());
|
|
![](https://seccdn.libravatar.org/avatar/f1a0ddfc363b14e167bf8548e95f340032d1964ae5b6ea4f235e351fd948008e?s=16&d=retro) |
161d5c0 |
+ cpumask_copy(mask, d->dirty_cpumask);
|
|
![](https://seccdn.libravatar.org/avatar/f1a0ddfc363b14e167bf8548e95f340032d1964ae5b6ea4f235e351fd948008e?s=16&d=retro) |
161d5c0 |
+
|
|
![](https://seccdn.libravatar.org/avatar/f1a0ddfc363b14e167bf8548e95f340032d1964ae5b6ea4f235e351fd948008e?s=16&d=retro) |
161d5c0 |
+ /* Don't flush if the timestamp is old enough */
|
|
![](https://seccdn.libravatar.org/avatar/f1a0ddfc363b14e167bf8548e95f340032d1964ae5b6ea4f235e351fd948008e?s=16&d=retro) |
161d5c0 |
+ tlbflush_filter(mask, page->tlbflush_timestamp);
|
|
![](https://seccdn.libravatar.org/avatar/f1a0ddfc363b14e167bf8548e95f340032d1964ae5b6ea4f235e351fd948008e?s=16&d=retro) |
161d5c0 |
+
|
|
![](https://seccdn.libravatar.org/avatar/f1a0ddfc363b14e167bf8548e95f340032d1964ae5b6ea4f235e351fd948008e?s=16&d=retro) |
161d5c0 |
+ if ( unlikely(!cpumask_empty(mask)) &&
|
|
![](https://seccdn.libravatar.org/avatar/f1a0ddfc363b14e167bf8548e95f340032d1964ae5b6ea4f235e351fd948008e?s=16&d=retro) |
161d5c0 |
+ /* Shadow mode: track only writable pages. */
|
|
![](https://seccdn.libravatar.org/avatar/f1a0ddfc363b14e167bf8548e95f340032d1964ae5b6ea4f235e351fd948008e?s=16&d=retro) |
161d5c0 |
+ (!shadow_mode_enabled(d) ||
|
|
![](https://seccdn.libravatar.org/avatar/f1a0ddfc363b14e167bf8548e95f340032d1964ae5b6ea4f235e351fd948008e?s=16&d=retro) |
161d5c0 |
+ ((nx & PGT_type_mask) == PGT_writable_page)) )
|
|
![](https://seccdn.libravatar.org/avatar/f1a0ddfc363b14e167bf8548e95f340032d1964ae5b6ea4f235e351fd948008e?s=16&d=retro) |
161d5c0 |
+ {
|
|
![](https://seccdn.libravatar.org/avatar/f1a0ddfc363b14e167bf8548e95f340032d1964ae5b6ea4f235e351fd948008e?s=16&d=retro) |
161d5c0 |
+ perfc_incr(need_flush_tlb_flush);
|
|
![](https://seccdn.libravatar.org/avatar/f1a0ddfc363b14e167bf8548e95f340032d1964ae5b6ea4f235e351fd948008e?s=16&d=retro) |
161d5c0 |
+ /*
|
|
![](https://seccdn.libravatar.org/avatar/f1a0ddfc363b14e167bf8548e95f340032d1964ae5b6ea4f235e351fd948008e?s=16&d=retro) |
161d5c0 |
+ * If page was a page table make sure the flush is
|
|
![](https://seccdn.libravatar.org/avatar/f1a0ddfc363b14e167bf8548e95f340032d1964ae5b6ea4f235e351fd948008e?s=16&d=retro) |
161d5c0 |
+ * performed using an IPI in order to avoid changing the
|
|
![](https://seccdn.libravatar.org/avatar/f1a0ddfc363b14e167bf8548e95f340032d1964ae5b6ea4f235e351fd948008e?s=16&d=retro) |
161d5c0 |
+ * type of a page table page under the feet of
|
|
![](https://seccdn.libravatar.org/avatar/f1a0ddfc363b14e167bf8548e95f340032d1964ae5b6ea4f235e351fd948008e?s=16&d=retro) |
161d5c0 |
+ * spurious_page_fault().
|
|
![](https://seccdn.libravatar.org/avatar/f1a0ddfc363b14e167bf8548e95f340032d1964ae5b6ea4f235e351fd948008e?s=16&d=retro) |
161d5c0 |
+ */
|
|
![](https://seccdn.libravatar.org/avatar/f1a0ddfc363b14e167bf8548e95f340032d1964ae5b6ea4f235e351fd948008e?s=16&d=retro) |
161d5c0 |
+ flush_mask(mask,
|
|
![](https://seccdn.libravatar.org/avatar/f1a0ddfc363b14e167bf8548e95f340032d1964ae5b6ea4f235e351fd948008e?s=16&d=retro) |
161d5c0 |
+ (x & PGT_type_mask) &&
|
|
![](https://seccdn.libravatar.org/avatar/f1a0ddfc363b14e167bf8548e95f340032d1964ae5b6ea4f235e351fd948008e?s=16&d=retro) |
161d5c0 |
+ (x & PGT_type_mask) <= PGT_root_page_table
|
|
![](https://seccdn.libravatar.org/avatar/f1a0ddfc363b14e167bf8548e95f340032d1964ae5b6ea4f235e351fd948008e?s=16&d=retro) |
161d5c0 |
+ ? FLUSH_TLB | FLUSH_FORCE_IPI
|
|
![](https://seccdn.libravatar.org/avatar/f1a0ddfc363b14e167bf8548e95f340032d1964ae5b6ea4f235e351fd948008e?s=16&d=retro) |
161d5c0 |
+ : FLUSH_TLB);
|
|
![](https://seccdn.libravatar.org/avatar/f1a0ddfc363b14e167bf8548e95f340032d1964ae5b6ea4f235e351fd948008e?s=16&d=retro) |
161d5c0 |
+ }
|
|
![](https://seccdn.libravatar.org/avatar/f1a0ddfc363b14e167bf8548e95f340032d1964ae5b6ea4f235e351fd948008e?s=16&d=retro) |
161d5c0 |
+ }
|
|
![](https://seccdn.libravatar.org/avatar/f1a0ddfc363b14e167bf8548e95f340032d1964ae5b6ea4f235e351fd948008e?s=16&d=retro) |
161d5c0 |
+ }
|
|
![](https://seccdn.libravatar.org/avatar/f1a0ddfc363b14e167bf8548e95f340032d1964ae5b6ea4f235e351fd948008e?s=16&d=retro) |
161d5c0 |
+
|
|
![](https://seccdn.libravatar.org/avatar/f1a0ddfc363b14e167bf8548e95f340032d1964ae5b6ea4f235e351fd948008e?s=16&d=retro) |
161d5c0 |
if ( unlikely(((x & PGT_type_mask) == PGT_writable_page) !=
|
|
![](https://seccdn.libravatar.org/avatar/f1a0ddfc363b14e167bf8548e95f340032d1964ae5b6ea4f235e351fd948008e?s=16&d=retro) |
161d5c0 |
(type == PGT_writable_page)) )
|
|
![](https://seccdn.libravatar.org/avatar/f1a0ddfc363b14e167bf8548e95f340032d1964ae5b6ea4f235e351fd948008e?s=16&d=retro) |
161d5c0 |
{
|
|
![](https://seccdn.libravatar.org/avatar/f1a0ddfc363b14e167bf8548e95f340032d1964ae5b6ea4f235e351fd948008e?s=16&d=retro) |
161d5c0 |
@@ -3120,13 +3126,25 @@ static int _get_page_type(struct page_info *page, unsigned long type,
|
|
![](https://seccdn.libravatar.org/avatar/f1a0ddfc363b14e167bf8548e95f340032d1964ae5b6ea4f235e351fd948008e?s=16&d=retro) |
161d5c0 |
|
|
![](https://seccdn.libravatar.org/avatar/f1a0ddfc363b14e167bf8548e95f340032d1964ae5b6ea4f235e351fd948008e?s=16&d=retro) |
161d5c0 |
if ( unlikely(!(nx & PGT_validated)) )
|
|
![](https://seccdn.libravatar.org/avatar/f1a0ddfc363b14e167bf8548e95f340032d1964ae5b6ea4f235e351fd948008e?s=16&d=retro) |
161d5c0 |
{
|
|
![](https://seccdn.libravatar.org/avatar/f1a0ddfc363b14e167bf8548e95f340032d1964ae5b6ea4f235e351fd948008e?s=16&d=retro) |
161d5c0 |
- if ( !(x & PGT_partial) )
|
|
![](https://seccdn.libravatar.org/avatar/f1a0ddfc363b14e167bf8548e95f340032d1964ae5b6ea4f235e351fd948008e?s=16&d=retro) |
161d5c0 |
+ /*
|
|
![](https://seccdn.libravatar.org/avatar/f1a0ddfc363b14e167bf8548e95f340032d1964ae5b6ea4f235e351fd948008e?s=16&d=retro) |
161d5c0 |
+ * No special validation needed for writable or shared pages. Page
|
|
![](https://seccdn.libravatar.org/avatar/f1a0ddfc363b14e167bf8548e95f340032d1964ae5b6ea4f235e351fd948008e?s=16&d=retro) |
161d5c0 |
+ * tables and GDT/LDT need to have their contents audited.
|
|
![](https://seccdn.libravatar.org/avatar/f1a0ddfc363b14e167bf8548e95f340032d1964ae5b6ea4f235e351fd948008e?s=16&d=retro) |
161d5c0 |
+ *
|
|
![](https://seccdn.libravatar.org/avatar/f1a0ddfc363b14e167bf8548e95f340032d1964ae5b6ea4f235e351fd948008e?s=16&d=retro) |
161d5c0 |
+ * per validate_page(), non-atomic updates are fine here.
|
|
![](https://seccdn.libravatar.org/avatar/f1a0ddfc363b14e167bf8548e95f340032d1964ae5b6ea4f235e351fd948008e?s=16&d=retro) |
161d5c0 |
+ */
|
|
![](https://seccdn.libravatar.org/avatar/f1a0ddfc363b14e167bf8548e95f340032d1964ae5b6ea4f235e351fd948008e?s=16&d=retro) |
161d5c0 |
+ if ( type == PGT_writable_page || type == PGT_shared_page )
|
|
![](https://seccdn.libravatar.org/avatar/f1a0ddfc363b14e167bf8548e95f340032d1964ae5b6ea4f235e351fd948008e?s=16&d=retro) |
161d5c0 |
+ page->u.inuse.type_info |= PGT_validated;
|
|
![](https://seccdn.libravatar.org/avatar/f1a0ddfc363b14e167bf8548e95f340032d1964ae5b6ea4f235e351fd948008e?s=16&d=retro) |
161d5c0 |
+ else
|
|
![](https://seccdn.libravatar.org/avatar/f1a0ddfc363b14e167bf8548e95f340032d1964ae5b6ea4f235e351fd948008e?s=16&d=retro) |
161d5c0 |
{
|
|
![](https://seccdn.libravatar.org/avatar/f1a0ddfc363b14e167bf8548e95f340032d1964ae5b6ea4f235e351fd948008e?s=16&d=retro) |
161d5c0 |
- page->nr_validated_ptes = 0;
|
|
![](https://seccdn.libravatar.org/avatar/f1a0ddfc363b14e167bf8548e95f340032d1964ae5b6ea4f235e351fd948008e?s=16&d=retro) |
161d5c0 |
- page->partial_flags = 0;
|
|
![](https://seccdn.libravatar.org/avatar/f1a0ddfc363b14e167bf8548e95f340032d1964ae5b6ea4f235e351fd948008e?s=16&d=retro) |
161d5c0 |
- page->linear_pt_count = 0;
|
|
![](https://seccdn.libravatar.org/avatar/f1a0ddfc363b14e167bf8548e95f340032d1964ae5b6ea4f235e351fd948008e?s=16&d=retro) |
161d5c0 |
+ if ( !(x & PGT_partial) )
|
|
![](https://seccdn.libravatar.org/avatar/f1a0ddfc363b14e167bf8548e95f340032d1964ae5b6ea4f235e351fd948008e?s=16&d=retro) |
161d5c0 |
+ {
|
|
![](https://seccdn.libravatar.org/avatar/f1a0ddfc363b14e167bf8548e95f340032d1964ae5b6ea4f235e351fd948008e?s=16&d=retro) |
161d5c0 |
+ page->nr_validated_ptes = 0;
|
|
![](https://seccdn.libravatar.org/avatar/f1a0ddfc363b14e167bf8548e95f340032d1964ae5b6ea4f235e351fd948008e?s=16&d=retro) |
161d5c0 |
+ page->partial_flags = 0;
|
|
![](https://seccdn.libravatar.org/avatar/f1a0ddfc363b14e167bf8548e95f340032d1964ae5b6ea4f235e351fd948008e?s=16&d=retro) |
161d5c0 |
+ page->linear_pt_count = 0;
|
|
![](https://seccdn.libravatar.org/avatar/f1a0ddfc363b14e167bf8548e95f340032d1964ae5b6ea4f235e351fd948008e?s=16&d=retro) |
161d5c0 |
+ }
|
|
![](https://seccdn.libravatar.org/avatar/f1a0ddfc363b14e167bf8548e95f340032d1964ae5b6ea4f235e351fd948008e?s=16&d=retro) |
161d5c0 |
+
|
|
![](https://seccdn.libravatar.org/avatar/f1a0ddfc363b14e167bf8548e95f340032d1964ae5b6ea4f235e351fd948008e?s=16&d=retro) |
161d5c0 |
+ rc = validate_page(page, type, preemptible);
|
|
![](https://seccdn.libravatar.org/avatar/f1a0ddfc363b14e167bf8548e95f340032d1964ae5b6ea4f235e351fd948008e?s=16&d=retro) |
161d5c0 |
}
|
|
![](https://seccdn.libravatar.org/avatar/f1a0ddfc363b14e167bf8548e95f340032d1964ae5b6ea4f235e351fd948008e?s=16&d=retro) |
161d5c0 |
- rc = validate_page(page, type, preemptible);
|
|
![](https://seccdn.libravatar.org/avatar/f1a0ddfc363b14e167bf8548e95f340032d1964ae5b6ea4f235e351fd948008e?s=16&d=retro) |
161d5c0 |
}
|
|
![](https://seccdn.libravatar.org/avatar/f1a0ddfc363b14e167bf8548e95f340032d1964ae5b6ea4f235e351fd948008e?s=16&d=retro) |
161d5c0 |
|
|
![](https://seccdn.libravatar.org/avatar/f1a0ddfc363b14e167bf8548e95f340032d1964ae5b6ea4f235e351fd948008e?s=16&d=retro) |
161d5c0 |
out:
|