Colin Cross [Sat, 14 Dec 2013 03:26:32 +0000 (19:26 -0800)]
ion: fix bugs in cma heap
Implement ion_cma_unmap_kernel, ion will call it unconditionally.
Use correct gfp flags when calling dma_alloc_coherent so it doesn't
try to use atomic DMA memory.
Check for invalid alignment when allocating.
Reject cached allocations - the cpu address returned by
dma_alloc_coherent is always going to be an uncached mapping, so
map_kernel will not see data written by a cached userspace mapping.
Signed-off-by: Colin Cross <ccross@android.com> Signed-off-by: John Stultz <john.stultz@linaro.org> Signed-off-by: Greg Kroah-Hartman <gregkh@linuxfoundation.org>
Colin Cross [Sat, 14 Dec 2013 03:26:30 +0000 (19:26 -0800)]
ion: add helper to zero contiguous region of pages
Add ion_heap_pages_zero for ion heaps to use to zero pages
during initialization or allocation, when a struct ion_buffer
may not be available. Use it from the chunk heap and carveout
heaps.
Signed-off-by: Colin Cross <ccross@android.com> Signed-off-by: John Stultz <john.stultz@linaro.org> Signed-off-by: Greg Kroah-Hartman <gregkh@linuxfoundation.org>
Colin Cross [Sat, 14 Dec 2013 03:26:29 +0000 (19:26 -0800)]
ion: carveout heap: zero buffers on free, fix memory leak
The carveout heap wasn't zeroing its buffers after use.
Create the sg_table during allocate instead of map_dma, to allow
using the sg_table during free, and call ion_heap_buffer_zero
during free. Also fixes a missing kfree when destroying the
table.
Signed-off-by: Colin Cross <ccross@android.com> Signed-off-by: John Stultz <john.stultz@linaro.org> Signed-off-by: Greg Kroah-Hartman <gregkh@linuxfoundation.org>
Colin Cross [Sat, 14 Dec 2013 03:26:27 +0000 (19:26 -0800)]
ion: use alloc_pages in system contig heap
There is no reason to use kzalloc, just call alloc_pages directly.
Change the GFP from GFP_KERNEL to include __GFP_HIGH, to allow it
to return contiguous pages from highmem. virt_to_* functions
aren't valid on highmem pages, so store the struct page * in an
sg_table in buffer->priv_virt like most other heaps, and replace
virt_to_* with page_to_*.
Signed-off-by: Colin Cross <ccross@android.com> Signed-off-by: John Stultz <john.stultz@linaro.org> Signed-off-by: Greg Kroah-Hartman <gregkh@linuxfoundation.org>
Colin Cross [Sat, 14 Dec 2013 03:26:26 +0000 (19:26 -0800)]
ion: allow cached mappings of chunk and system heap buffers
Now that ion_vm_fault uses vm_insert_pfn instead of vm_insert_page
cached buffers can be supported in any heap. Remove the checks
in the chunk and system heaps.
Signed-off-by: Colin Cross <ccross@android.com> Signed-off-by: John Stultz <john.stultz@linaro.org> Signed-off-by: Greg Kroah-Hartman <gregkh@linuxfoundation.org>
Colin Cross [Sat, 14 Dec 2013 03:26:25 +0000 (19:26 -0800)]
ion: remove ion_heap_alloc_pages
Now that ion_vm_fault doesn't need a struct page with a nonzero
refcount, there is no need allocate heap memory for cached pages using
split_page. Remove the ion_heap_alloc_pages and ion_heap_free_pages
helpers in favor of direct calls to alloc_pages and __free_pages,
and remove the special handling in the system heap.
Signed-off-by: Colin Cross <ccross@android.com> Signed-off-by: John Stultz <john.stultz@linaro.org> Signed-off-by: Greg Kroah-Hartman <gregkh@linuxfoundation.org>
Colin Cross [Sat, 14 Dec 2013 03:26:24 +0000 (19:26 -0800)]
ion: use vm_insert_pfn for faulted pages
Most ion userspace mappings are created with remap_pfn_range. Use
vm_insert_pfn instead of vm_insert_page to make faulted cached
mappings look more like uncached mappings.
Signed-off-by: Colin Cross <ccross@android.com> Signed-off-by: John Stultz <john.stultz@linaro.org> Signed-off-by: Greg Kroah-Hartman <gregkh@linuxfoundation.org>
Colin Cross [Sat, 14 Dec 2013 03:26:21 +0000 (19:26 -0800)]
ion: optimize ion_heap_buffer_zero
ion_heap_buffer_zero can spend a long time in unmap_kernel_range
if it has to broadcast a tlb flush to every cpu for every page.
Modify it to batch pages into a larger region to clear using a
single mapping. This may cause the mapping size to change if
the buffer size is not a multiple of the mapping size, so
switch to allocating the address space for each chunk. This
allows us to use vm_map_ram to handle the allocation and mapping
together.
The number of pages to zero using a single mapping is set to 32
to hit the fastpath in vm_map_ram.
Signed-off-by: Colin Cross <ccross@android.com> Signed-off-by: John Stultz <john.stultz@linaro.org> Signed-off-by: Greg Kroah-Hartman <gregkh@linuxfoundation.org>
John Stultz [Sat, 14 Dec 2013 03:26:18 +0000 (19:26 -0800)]
ion: Fix two small issues in system_heap allocation
In testing ion system heap allocations, I ran across two issues:
1) Not k*z*allocing the sg table. This can cause trouble if
we end up trying call sg_alloc_table() with too many entries,
then sg_alloc_table() internally fails and tries to free what it
thinks is internal table structure, which causes bad pointer
traversals.
2) The second list_for_each_entry probably should be _safe,
since I was seeing strange lock warnings and oopses on occasion.
This seems to resolve it, but could use some extra checking.
Signed-off-by: John Stultz <john.stultz@linaro.org> Signed-off-by: Greg Kroah-Hartman <gregkh@linuxfoundation.org>
Colin Cross [Sat, 14 Dec 2013 03:26:16 +0000 (19:26 -0800)]
ion: clean up ioctls
Convert the ion ioctls to use _IOW instead of _IOWR where
appropriate, and factor out the copy_from_user and copy_to_user
based on the _IOC_DIR bits. For the existing incorrect ioctls,
add a function to wrap _IOC_DIR to return the corrected value.
Signed-off-by: Colin Cross <ccross@android.com> Signed-off-by: John Stultz <john.stultz@linaro.org> Signed-off-by: Greg Kroah-Hartman <gregkh@linuxfoundation.org>
Colin Cross [Sat, 14 Dec 2013 03:26:13 +0000 (19:26 -0800)]
ion: don't use phys_to_page or __phys_to_pfn
phys_to_page and __phys_to_pfn don't exist on all platforms.
Use a combination of pfn_to_page, PFN_DOWN, page_to_pfn, and
virt_to_page to get the same results.
Signed-off-by: Colin Cross <ccross@android.com> Signed-off-by: John Stultz <john.stultz@linaro.org> Signed-off-by: Greg Kroah-Hartman <gregkh@linuxfoundation.org>
Colin Cross [Fri, 13 Dec 2013 22:25:06 +0000 (14:25 -0800)]
ion: don't use __arm_ioremap to map pages
ion_heap_map_kernel already implements mapping a scatterlist of
pages into the kernel, and all heaps are required to have struct
pages associated with them, so delete the functions that use
__arm_ioremap and use ion_heap_map_kernel instead.
Signed-off-by: Colin Cross <ccross@android.com> Signed-off-by: John Stultz <john.stultz@linaro.org> Signed-off-by: Greg Kroah-Hartman <gregkh@linuxfoundation.org>
Colin Cross [Fri, 13 Dec 2013 22:25:04 +0000 (14:25 -0800)]
ion: add test device for unit tests to interact with dma_bufs
Add a /dev/ion-test device that will be created if CONFIG_ION_TEST
is set. The device accepts a dma_buf fd and allows reading and
writing to the backing memory using DMA-like apis or kernel mapping
apis. Can be used to test the dma_buf mapping ops, including
the ion implementations, from userspace.
Signed-off-by: Colin Cross <ccross@android.com> Signed-off-by: John Stultz <john.stultz@linaro.org> Signed-off-by: Greg Kroah-Hartman <gregkh@linuxfoundation.org>
Colin Cross [Fri, 13 Dec 2013 22:25:03 +0000 (14:25 -0800)]
ion: check invalid values in ion_system_heap
ion_system_heap can only satisfy page alignment, and
ion_system_contig_heap can only satisify alignment to the
allocation size. Neither can support faulting user mappings
because they use slab pages.
Signed-off-by: Colin Cross <ccross@android.com> Signed-off-by: John Stultz <john.stultz@linaro.org> Signed-off-by: Greg Kroah-Hartman <gregkh@linuxfoundation.org>
Colin Cross [Fri, 13 Dec 2013 22:25:01 +0000 (14:25 -0800)]
ion: fix dma APIs
__dma_page_cpu_to_dev is a private ARM api that is not available
on 3.10 and was never available on other architectures. We can
get the same behavior by calling dma_sync_sg_for_device with a
scatterlist containing a single page. It's still not quite a
kosher use of the dma apis, we still conflate physical addresses
with bus addresses, but it should at least compile on all
platforms, and work on any platform that doesn't have a physical
to bus address translation.
Signed-off-by: Colin Cross <ccross@android.com> Signed-off-by: John Stultz <john.stultz@linaro.org> Signed-off-by: Greg Kroah-Hartman <gregkh@linuxfoundation.org>
Colin Cross [Fri, 13 Dec 2013 22:25:00 +0000 (14:25 -0800)]
ion: fix crash when alloc len is -1
If userspace passes a length between -4095 and -1 to allocate it
will pass the len != 0 check, but when len is page aligned it will
be 0. Check len after page aligning.
Drop the warning as well, userspace shouldn't be able to trigger
a warning in the kernel.
Signed-off-by: Colin Cross <ccross@android.com> Signed-off-by: John Stultz <john.stultz@linaro.org> Signed-off-by: Greg Kroah-Hartman <gregkh@linuxfoundation.org>
Colin Cross [Fri, 13 Dec 2013 22:24:59 +0000 (14:24 -0800)]
ion: hold reference to handle after ion_uhandle_get
commit 1262ab1846cf76f7549c66ef709120dbfbe6d49f (ion: replace
userspace handle cookies with idr) broke the locking in ion.
The ION_IOC_FREE and ION_IOC_MAP ioctls were relying on
ion_handle_validate to detect the case where a call raced
with another ION_IOC_FREE which may have freed the struct
ion_handle.
Rename ion_uhandle_get to ion_handle_get_by_id, and have it
take the client lock and return with an extra reference to
the handle. Make each caller put its reference once it
is done with the handle.
Also modify users of ion_handle_validate to continue to hold
the client lock after calling ion_handle_validate until
they are done with the handle, and warn if ion_handle_validate
is called without the client lock held.
Signed-off-by: Colin Cross <ccross@android.com> Signed-off-by: John Stultz <john.stultz@linaro.org> Signed-off-by: Greg Kroah-Hartman <gregkh@linuxfoundation.org>
John Stultz [Fri, 13 Dec 2013 22:24:58 +0000 (14:24 -0800)]
ion: Fix compat support to use proper compat ioctl numbers
The compat support added to ion didn't provide compat ioctl numbers
(who's value depends on the compat structure size). So 32bit
applications would get an error when trying to make ioctl calls.
This patch adds the needed COMPAT_ macros and uses them in the
compat_ion_ioctl, translating them to their non-compat cmd when
calling the normal ioctl call.
Signed-off-by: John Stultz <john.stultz@linaro.org> Signed-off-by: Greg Kroah-Hartman <gregkh@linuxfoundation.org>
Rom Lemarchand [Fri, 13 Dec 2013 22:24:55 +0000 (14:24 -0800)]
ion: add compat_ioctl
Add a compat_ioctl to the ion driver
Signed-off-by: Rom Lemarchand <romlem@google.com>
[jstultz: modified patch to apply to staging directory] Signed-off-by: John Stultz <john.stultz@linaro.org> Signed-off-by: Greg Kroah-Hartman <gregkh@linuxfoundation.org>
Rom Lemarchand [Fri, 13 Dec 2013 22:24:54 +0000 (14:24 -0800)]
ion: change ion_user_handle_t definition to int
Turn ion_user_handle_t to int. This change reflects the underlying type
returned by the ion driver.
Signed-off-by: Rom Lemarchand <romlem@google.com>
[jstultz: modified patch to apply to staging directory] Signed-off-by: John Stultz <john.stultz@linaro.org> Signed-off-by: Greg Kroah-Hartman <gregkh@linuxfoundation.org>
Rom Lemarchand [Fri, 13 Dec 2013 22:24:53 +0000 (14:24 -0800)]
ion: add new ion_user_handle_t type for the user-space token
Declare new ion_user_handle_t type to contain the token returned to user-space.
This allows a 2-step migration of the user-space code to a new kernel header
first, then will allow us to change the definition of the ion_user_handle_type_t
to int without breaking the API.
Signed-off-by: Rom Lemarchand <romlem@google.com>
(cherry picked from commit ebb8269bbb05b06ecedca3e21b3e65f23d48eadd)
[jstultz: modified patch to apply to staging directory] Signed-off-by: John Stultz <john.stultz@linaro.org> Signed-off-by: Greg Kroah-Hartman <gregkh@linuxfoundation.org>
Colin Cross [Fri, 13 Dec 2013 22:24:52 +0000 (14:24 -0800)]
ion: don't use id 0 for handle cookie
ion userspace clients think that the cookie is a pointer, so they
use NULL to check if the handle has been initialized. Set the first
id number to 1.
Signed-off-by: Colin Cross <ccross@android.com>
[jstultz: modified patch to apply to staging directory] Signed-off-by: John Stultz <john.stultz@linaro.org> Signed-off-by: Greg Kroah-Hartman <gregkh@linuxfoundation.org>
Colin Cross [Fri, 13 Dec 2013 22:24:51 +0000 (14:24 -0800)]
ion: index client->handles rbtree by buffer
The only remaining users of the client->handles rbtree are
iterating through it like a list. Keep the rbtree, but change
its index to be the buffer address instead of the handle address,
which makes ion_handle_lookup a fast rbtree search.
Signed-off-by: Colin Cross <ccross@android.com>
[jstultz: modified patch to apply to staging directory] Signed-off-by: John Stultz <john.stultz@linaro.org> Signed-off-by: Greg Kroah-Hartman <gregkh@linuxfoundation.org>
Colin Cross [Fri, 13 Dec 2013 22:24:50 +0000 (14:24 -0800)]
ion: replace userspace handle cookies with idr
Userspace handles should not leak kernel virtual addresses to
userspace. They have to be validated by looking them up in an
rbtree anyways, so replace them with an idr and validate them
by using idr_find to convert the id number to the struct
ion_handle pointer.
Signed-off-by: Colin Cross <ccross@android.com>
[jstultz: modified patch to apply to staging directory] Signed-off-by: John Stultz <john.stultz@linaro.org> Signed-off-by: Greg Kroah-Hartman <gregkh@linuxfoundation.org>
Colin Cross [Fri, 13 Dec 2013 22:24:49 +0000 (14:24 -0800)]
ion: remove IS_ERR_OR_NULL
IS_ERR_OR_NULL is often part of a bad pattern that can accidentally
return 0 on error:
if (IS_ERR_OR_NULL(ptr))
return PTR_ERR(ptr);
It also usually means that the errors of a function are not well
defined. Replace all uses in ion.c by ensure that the return
type of any function in ion is an ERR_PTR.
Specify that the expected return value from map_kernel or map_dma
heap ops is ERR_PTR, and warn if a heap returns NULL.
Signed-off-by: Colin Cross <ccross@android.com>
[jstultz: modified patch to apply to staging directory] Signed-off-by: John Stultz <john.stultz@linaro.org> Signed-off-by: Greg Kroah-Hartman <gregkh@linuxfoundation.org>
Colin Cross [Fri, 13 Dec 2013 22:24:48 +0000 (14:24 -0800)]
ion: convert map_kernel to return ERR_PTR
ion is going to stop accepting NULL as an error value, use ERR_PTR.
Signed-off-by: Colin Cross <ccross@android.com>
[jstultz: modified patch to apply to staging directory] Signed-off-by: John Stultz <john.stultz@linaro.org> Signed-off-by: Greg Kroah-Hartman <gregkh@linuxfoundation.org>
Colin Cross [Fri, 13 Dec 2013 22:24:47 +0000 (14:24 -0800)]
ion: add free list size to heap debug files
Signed-off-by: Colin Cross <ccross@android.com>
[jstultz: modified patch to apply to staging directory] Signed-off-by: John Stultz <john.stultz@linaro.org> Signed-off-by: Greg Kroah-Hartman <gregkh@linuxfoundation.org>
Colin Cross [Fri, 13 Dec 2013 22:24:46 +0000 (14:24 -0800)]
ion: chunk_heap: fix leak in allocated counter
buffer->size is controlled by the outer ion layer, don't modify it
inside the heap. Instead, compute the rounded up allocated size
on demand.
Signed-off-by: Colin Cross <ccross@android.com>
[jstultz: modified patch to apply to staging directory] Signed-off-by: John Stultz <john.stultz@linaro.org> Signed-off-by: Greg Kroah-Hartman <gregkh@linuxfoundation.org>
Previously the code to fault ion buffers in one page at a time had a
performance problem caused by the requirement to traverse the sg list
looking for the right page to load in (a result of the fact that the items in
the list may not be of uniform size). To fix the problem, for buffers
that will be faulted in, also keep a flat array of all the pages in the buffer
to use from the fault handler. To recover some of the additional memory
footprint this creates per buffer, dirty bits used to indicate which
pages have been faulted in to the cpu are now stored in the low bit of each
page struct pointer in the page array.
Signed-off-by: Rebecca Schultz Zavin <rebecca@android.com>
[jstultz: modified patch to apply to staging directory] Signed-off-by: John Stultz <john.stultz@linaro.org> Signed-off-by: Greg Kroah-Hartman <gregkh@linuxfoundation.org>
New heap type ION_HEAP_TYPE_DMA where allocation is done with dma_alloc_coherent API.
device coherent_dma_mask must be set to DMA_BIT_MASK(32).
ion_platform_heap private field is used to retrieve the device linked to CMA,
if NULL the default CMA area is used.
ion_cma_get_sgtable is a copy of dma_common_get_sgtable function which should
be in kernel 3.5
Signed-off-by: Benjamin Gaignard <benjamin.gaignard@linaro.org>
[jstultz: modified patch to apply to staging directory] Signed-off-by: John Stultz <john.stultz@linaro.org> Signed-off-by: Greg Kroah-Hartman <gregkh@linuxfoundation.org>
fix ion_platform_heap to make is use an usual way in board configuration file.
Signed-off-by: Benjamin Gaignard <benjamin.gaignard@linaro.org>
[jstultz: modified patch to apply to staging directory] Signed-off-by: John Stultz <john.stultz@linaro.org> Signed-off-by: Greg Kroah-Hartman <gregkh@linuxfoundation.org>
John Stultz [Fri, 13 Dec 2013 22:24:42 +0000 (14:24 -0800)]
ion: Add Kconfig dependency to ARM
The ion code has some very specific arm-isms which keeps it
from building on other architectures. These should probably be
resolved, but in the mean time, add a dependency on CONFIG_ARM
to avoid build failures.
v2: Fix earlier flub, sending out an early untested version of
the patch.
Cc: Arve Hjønnevåg <arve@android.com> Cc: Rebecca Schultz Zavin <rebecca@android.com> Cc: Android Kernel Team <kernel-team@android.com>
[jstultz: modified patch to apply to staging directory] Signed-off-by: John Stultz <john.stultz@linaro.org> Signed-off-by: Greg Kroah-Hartman <gregkh@linuxfoundation.org>
gpu: ion: Also shrink memory cached in the deferred free list
When the system is low on memory, we want to shrink any cached
system memory ion is holding. Previously we were shrinking memory
in the page pools, but not in the deferred free list. This patch
makes it possible to shrink both. It also moves the shrinker
code into the heaps so they can correctly manage any caches they
might contain.
Signed-off-by: Rebecca Schultz Zavin <rebecca@android.com>
[jstultz: modified patch to apply to staging directory] Signed-off-by: John Stultz <john.stultz@linaro.org> Signed-off-by: Greg Kroah-Hartman <gregkh@linuxfoundation.org>
gpu: ion: ion_chunk_heap: Zero chunk heap memory at creation time
Allocations from the ion heap need to be zeroed to protect userspace
from seeing memory belonging to other processes. First allocations
from this heap were not zero'd allowing users to see memory from other
processes on a warm reset.
Signed-off-by: Rebecca Schultz Zavin <rebecca@android.com>
[jstultz: modified patch to apply to staging directory] Signed-off-by: John Stultz <john.stultz@linaro.org> Signed-off-by: Greg Kroah-Hartman <gregkh@linuxfoundation.org>
Add the ability for a heap to free buffers asynchrounously. Freed buffers
are placed on a free list and freed from a low priority background thread.
If allocations from a particular heap fail, the free list is drained. This
patch also enable asynchronous frees from the chunk heap.
Signed-off-by: Rebecca Schultz Zavin <rebecca@android.com>
[jstultz: modified patch to apply to staging directory] Signed-off-by: John Stultz <john.stultz@linaro.org> Signed-off-by: Greg Kroah-Hartman <gregkh@linuxfoundation.org>
Johan Mossberg [Fri, 13 Dec 2013 22:24:34 +0000 (14:24 -0800)]
gpu: ion: Add support for sharing buffers with dma buf kernel handles
Currently ion can only share buffers with dma buf fd's. Fd's can not be
used inside the kernel as they are process specific so support for
sharing buffers with dma buf kernel handles is needed to support kernel
only use cases. An example use case could be a GPU driver using ion
that wants to share its output buffers with a 3d party display
controller driver supporting dma buf.
Signed-off-by: Johan Mossberg <johan.mossberg@stericsson.com>
[jstultz: modified patch to apply to staging directory] Signed-off-by: John Stultz <john.stultz@linaro.org> Signed-off-by: Greg Kroah-Hartman <gregkh@linuxfoundation.org>
Refactor the code in the system heap used to map and zero the buffers
into a seperate utility so it can be called from other heaps. Use it from
the chunk heap.
Signed-off-by: Rebecca Schultz Zavin <rebecca@android.com>
[jstultz: modified patch to apply to staging directory] Signed-off-by: John Stultz <john.stultz@linaro.org> Signed-off-by: Greg Kroah-Hartman <gregkh@linuxfoundation.org>
gpu: ion: Modify zeroing code so it only allocates address space once
vmap/vunmap spend a significant amount of time allocating the
address space to map into. Rather than allocating address space
for each page, instead allocate once for the entire allocation
and then just map and unmap each page into that address space.
Signed-off-by: Rebecca Schultz Zavin <rschultz@google.com>
[jstultz: modified patch to apply to staging directory] Signed-off-by: John Stultz <john.stultz@linaro.org> Signed-off-by: Greg Kroah-Hartman <gregkh@linuxfoundation.org>
Johan Mossberg [Fri, 13 Dec 2013 22:24:29 +0000 (14:24 -0800)]
gpu: ion: Export ion_client_create
Will enable modules to allocate memory with ion.
Signed-off-by: Johan Mossberg <johan.mossberg@stericsson.com>
[jstultz: modified patch to apply to staging directory] Signed-off-by: John Stultz <john.stultz@linaro.org> Signed-off-by: Greg Kroah-Hartman <gregkh@linuxfoundation.org>
gpu: ion: Clarify variable names and comments around heap ids v types
There is some confusion between when to use the heap type and when
the id. This patch clarifies this by using clearer variable names
and describing the intention in the comments. Also fixes the client
debug code to print heaps by id instead of type.
Signed-off-by: Rebecca Schultz Zavin <rebecca@android.com>
[jstultz: modified patch to apply to staging directory] Signed-off-by: John Stultz <john.stultz@linaro.org> Signed-off-by: Greg Kroah-Hartman <gregkh@linuxfoundation.org>
This patch adds support for a chunk heap that allows for buffers that are
made up of a list of fixed size chunks taken from a carveout. Chunk sizes
are configured when the heaps are created by passing the chunk size in the
priv field of the heap platform data.
Signed-off-by: Rebecca Schultz Zavin <rebecca@android.com>
[jstultz: modified patch to apply to staging directory] Signed-off-by: John Stultz <john.stultz@linaro.org> Signed-off-by: Greg Kroah-Hartman <gregkh@linuxfoundation.org>
gpu: ion: Refactor common mapping functions out of system heap
The system heap contained several general purpose functions to map
buffers to the kernel and userspace. This patch refactors those
into ion_heap.c so they can be used by other heaps.
Signed-off-by: Rebecca Schultz Zavin <rebecca@android.com>
[jstultz: modified patch to apply to staging directory] Signed-off-by: John Stultz <john.stultz@linaro.org> Signed-off-by: Greg Kroah-Hartman <gregkh@linuxfoundation.org>
Switches the rbtree tree of heaps for a plist. This significantly
simplifies the code and the list is small and is modified only at
first boot so the rbtree is unnecessary. This also switches
the traversal of the heap list to traverse from highest to lowest
id's. This allows allocations to pass a heap mask that falls
back on the system heap -- typically id 0, which is the common case.
Signed-off-by: Rebecca Schultz Zavin <rebecca@android.com>
[jstultz: modified patch to apply to staging directory] Signed-off-by: John Stultz <john.stultz@linaro.org> Signed-off-by: Greg Kroah-Hartman <gregkh@linuxfoundation.org>
gpu: ion: Modify reserve function for carveouts with no start address
This patch allows you to specify a heap that requires carveout memory
but that doesn't specify a start address. Memblock_alloc will be called
to find a location for these heaps.
Signed-off-by: Rebecca Schultz Zavin <rebecca@android.com>
[jstultz: modified patch to apply to staging directory] Signed-off-by: John Stultz <john.stultz@linaro.org> Signed-off-by: Greg Kroah-Hartman <gregkh@linuxfoundation.org>
use atomic_read to get the refcount value to avoid compilation warning
Signed-off-by: Benjamin Gaignard <benjamin.gaignard@linaro.org>
[jstultz: modified patch to apply to staging directory] Signed-off-by: John Stultz <john.stultz@linaro.org> Signed-off-by: Greg Kroah-Hartman <gregkh@linuxfoundation.org>
when using carveout heap ion_buffer_create function failed because
map_dma and unmap_dma operations aren't set by carveout heap.
Signed-off-by: Benjamin Gaignard <benjamin.gaignard@linaro.org>
[jstultz: modified patch to apply to staging directory] Signed-off-by: John Stultz <john.stultz@linaro.org> Signed-off-by: Greg Kroah-Hartman <gregkh@linuxfoundation.org>
When the requested mmap length was not an integer number of
chunks or the buffer, or if an offset was provided, a bug
would cause extra or incorrect pages of the buffer to be mapped.
Signed-off-by: Rebecca Schultz Zavin <rebecca@android.com>
[jstultz: modified patch to apply to staging directory] Signed-off-by: John Stultz <john.stultz@linaro.org> Signed-off-by: Greg Kroah-Hartman <gregkh@linuxfoundation.org>
gpu: ion: Clear GFP_WAIT flag on high order allocations
This will prevent the kernel from kicking off compaction
when higher order allocations are made. Instead we will
get these high order allocations only if they are readily
available.
Signed-off-by: Rebecca Schultz Zavin <rebecca@android.com>
[jstultz: modified patch to apply to staging directory] Signed-off-by: John Stultz <john.stultz@linaro.org> Signed-off-by: Greg Kroah-Hartman <gregkh@linuxfoundation.org>
Removes contention for lock between allocate and free by reducing
the length of time the lock is held for. Split out a seperate
lock to protect the list of heaps and replace it with a rwsem since
the list will most likely only be updated during initialization.
Signed-off-by: Rebecca Schultz Zavin <rebecca@android.com>
[jstultz: modified patch to apply to staging directory] Signed-off-by: John Stultz <john.stultz@linaro.org> Signed-off-by: Greg Kroah-Hartman <gregkh@linuxfoundation.org>
Currently the mutex is held while kmalloc is called, under a low memory
condition this might trigger the shrinker which also takes this mutex.
Refactor so the mutex is not held during allocation.
Signed-off-by: Rebecca Schultz Zavin <rebecca@android.com>
[jstultz: modified patch to apply to staging directory] Signed-off-by: John Stultz <john.stultz@linaro.org> Signed-off-by: Greg Kroah-Hartman <gregkh@linuxfoundation.org>
Split out low and high mem pages so they are correctly reported
when the shrinker is called.
Fix potential deadlock caused by holding the page pool lock while
allocationg and also needing that lock from the shrink function
Signed-off-by: Rebecca Schultz Zavin <rebecca@android.com>
[jstultz: modified patch to apply to staging directory] Signed-off-by: John Stultz <john.stultz@linaro.org> Signed-off-by: Greg Kroah-Hartman <gregkh@linuxfoundation.org>
When allocations larger than order 4 are made, use _GFP_NORETRY
and __GFP_NO_KSWAPD so kswapd doesn't get kicked off to reclaim
these larger chunks. For smaller allocaitons, these are
unnecessary, as the system should be able to reclaim these.
Signed-off-by: Rebecca Schultz Zavin <rebecca@android.com>
[jstultz: modified patch to apply to staging directory] Signed-off-by: John Stultz <john.stultz@linaro.org> Signed-off-by: Greg Kroah-Hartman <gregkh@linuxfoundation.org>
gpu: ion: Use the ion_page_pool from the system heap
With this change the system heap will use pagepools to avoid
having to invalidate memory when it is allocated, a
significant performance improvement on some systems.
Signed-off-by: Rebecca Schultz Zavin <rebecca@android.com>
[jstultz: modified patch to apply to staging directory] Signed-off-by: John Stultz <john.stultz@linaro.org> Signed-off-by: Greg Kroah-Hartman <gregkh@linuxfoundation.org>
This patch adds a new utility heaps can use to manage
memory. In the past we have found it can be very
expensive to manage the caches when allocating memory,
but it is imposible to know whether a previous user of a
given memory allocation had a cached mapping. This patch
adds the ability to store a pool of pages that were
previously used uncached so that cache maintenance
only need be done when growing this pool. The pool also
contains a shrinker so memory from the pool can be
recovered in low memory conditions.
Signed-off-by: Rebecca Schultz Zavin <rebecca@android.com>
[jstultz: modified patch to apply to staging directory] Signed-off-by: John Stultz <john.stultz@linaro.org> Signed-off-by: Greg Kroah-Hartman <gregkh@linuxfoundation.org>
gpu: ion: use vmalloc to allocate page array to map kernel
When ion_map_kernel is execute the system must allocate
an array large enough to hold a pointer to each page in
the buffer. If the buffer is very large and the system
memory has become very fragmented, there may not be
sufficient high order allocations available from kmalloc.
Use vmalloc instead.
Signed-off-by: Rebecca Schultz Zavin <rebecca@android.com>
[jstultz: modified patch to apply to staging directory] Signed-off-by: John Stultz <john.stultz@linaro.org> Signed-off-by: Greg Kroah-Hartman <gregkh@linuxfoundation.org>
gpu: ion: Stop trying to allocate from an order on first failure
With this patch the system heap will only try to allocate from each
order as long as allocations succeed. If it failes to obtain a higher
order allocation, it doesn't retry that order.
Signed-off-by: Rebecca Schultz Zavin <rebecca@android.com>
[jstultz: modified patch to apply to staging directory] Signed-off-by: John Stultz <john.stultz@linaro.org> Signed-off-by: Greg Kroah-Hartman <gregkh@linuxfoundation.org>
gpu: ion: optimize system heap for non fault buffers
If a buffer's user mappings are not going to be faulted
in it need not be allocated page wise. We can optimize
this common case by allocating an sglist of larger chunks
rather than creating an entry for each page in the
allocation.
Signed-off-by: Rebecca Schultz Zavin <rebecca@android.com>
[jstultz: modified patch to apply to staging directory] Signed-off-by: John Stultz <john.stultz@linaro.org> Signed-off-by: Greg Kroah-Hartman <gregkh@linuxfoundation.org>
gpu: ion: Add support for cached mappings that don't fault
We have found that faulting in the mappings for cached
allocations has a significant performance impact and is
only a benefit if only a small part of the buffer is
touched by the cpu (an uncommon case for software rendering).
This patch introduces a ION_FLAG_CACHED_NEEDS_SYNC
which determines whether a mapping should be created by
faulting or at mmap time. If this flag is set,
userspace must manage the caches explictly using the SYNC ioctl.
Signed-off-by: Rebecca Schultz Zavin <rebecca@android.com>
[jstultz: modified patch to apply to staging directory] Signed-off-by: John Stultz <john.stultz@linaro.org> Signed-off-by: Greg Kroah-Hartman <gregkh@linuxfoundation.org>
gpu: ion: Add debug information for orphaned handles
It is possible for a buffer to exist only as a dma_buf file
descriptor without it being held in any handles. When this
occurs it is impossible to track where the buffer is in the
system (without traversing every process in the system and
inspecting its file table). When buffers are orphaned like
this, copy the task comm and pid of the last client to hold
them into the buffer so we have a debugging hint as to where
this buffer came from. In practice this will probalby be
the process that allocated the buffer.
Signed-off-by: Rebecca Schultz Zavin <rebecca@android.com>
[jstultz: modified patch to apply to staging directory] Signed-off-by: John Stultz <john.stultz@linaro.org> Signed-off-by: Greg Kroah-Hartman <gregkh@linuxfoundation.org>
gpu: ion: Fix race between ion_import and ion_free
If preemted during ion_free after the refcount is updated but
before the handle can be removed from the rb_tree, import
might find that handle in the tree and try to reuse it
when execution returns to free, the handle will be cleaned
up leaving the caller of import with a corrupt handle.
This patch modifies the locking to protect agains this race.
Signed-off-by: Rebecca Schultz Zavin <rebecca@android.com>
[jstultz: modified patch to apply to staging directory] Signed-off-by: John Stultz <john.stultz@linaro.org> Signed-off-by: Greg Kroah-Hartman <gregkh@linuxfoundation.org>
gpu: ion: Switch to using kmalloc rather than kmap during allocation
Previously, metadata was stored in the allocated pages themselves
during allocation. However the system can only have a limited
number of kmapped pages. A very large allocation might exceed
this limit.
Signed-off-by: Rebecca Schultz Zavin <rebecca@android.com>
[jstultz: modified patch to apply to staging directory] Signed-off-by: John Stultz <john.stultz@linaro.org> Signed-off-by: Greg Kroah-Hartman <gregkh@linuxfoundation.org>
Laura Abbott [Fri, 13 Dec 2013 22:23:57 +0000 (14:23 -0800)]
gpu: ion: Don't call ion_buffer_put on error path
If dma_buf_fd fails, the dma_buf needs to be cleaned up by
calling dma_buf_put. dma_buf_put will call ion_dma_buf_release
which in turn calls ion_buffer_put to clean up the buffer
reference. Calling ion_buffer_put after dma_buf_put drops the
reference count by one more which is incorrect. Fix this by
getting rid of the extra ion_buffer_put call.
Signed-off-by: Laura Abbott <lauraa@codeaurora.org>
[jstultz: modified patch to apply to staging directory] Signed-off-by: John Stultz <john.stultz@linaro.org> Signed-off-by: Greg Kroah-Hartman <gregkh@linuxfoundation.org>
Olav Haugan [Fri, 13 Dec 2013 22:23:56 +0000 (14:23 -0800)]
gpu: ion: IOCTL return success when error occurs
ION_IOC_MAP, ION_IOC_SHARE, and ION_IOC_IMPORT may return
success when an error occurs.
Add correct error handling to ION_IOC_MAP, ION_IOC_SHARE, and
ION_IOC_IMPORT.
Signed-off-by: Olav Haugan <ohaugan@codeaurora.org>
[jstultz: modified patch to apply to staging directory] Signed-off-by: John Stultz <john.stultz@linaro.org> Signed-off-by: Greg Kroah-Hartman <gregkh@linuxfoundation.org>
Olav Haugan [Fri, 13 Dec 2013 22:23:55 +0000 (14:23 -0800)]
gpu: ion: Add EXPORT_SYMBOL to functions
Several functions in the ion interface is missing
EXPORT_SYMBOL. This is needed to allow clients to
use these functions from kernel modules.
Add EXPORT_SYMBOL to functions that are supposed
to be exposed.
Signed-off-by: Olav Haugan <ohaugan@codeaurora.org>
[jstultz: modified patch to apply to staging directory] Signed-off-by: John Stultz <john.stultz@linaro.org> Signed-off-by: Greg Kroah-Hartman <gregkh@linuxfoundation.org>
Olav Haugan [Fri, 13 Dec 2013 22:23:54 +0000 (14:23 -0800)]
gpu: ion: Add missing argument to WARN call
The condition argument to the WARN call in ion_free and
ion_share_dma_buf are missing. Add the argument to
allow correct printing of warning message.
Signed-off-by: Olav Haugan <ohaugan@codeaurora.org>
[jstultz: modified patch to apply to staging directory] Signed-off-by: John Stultz <john.stultz@linaro.org> Signed-off-by: Greg Kroah-Hartman <gregkh@linuxfoundation.org>
This is deprecated in favor of using the dma_buf api which will
automatically sync a buffer to memory when it is mapped to a device.
However, that functionality is not ready, so this patch adds the
ability to sync a buffer explicitly.
Signed-off-by: Rebecca Schultz Zavin <rebecca@android.com>
[jstultz: modified patch to apply to staging directory] Signed-off-by: John Stultz <john.stultz@linaro.org> Signed-off-by: Greg Kroah-Hartman <gregkh@linuxfoundation.org>
This patch adds cache maintenance operations to ion. As per mailing
list discussions regarding dma_buf, cache operations are done implicitly.
At buffer allocaiton time the user can select whether he'd like mappings
(both kernel and user) to be cached. When cached mappings are selected,
no mappings will be created for a buffer at mmap time. Instead pages will
be faulted in one at a time so we can track which pages require flushing
before dma. When the buffers are mapped for dma (via the dma_buf apis)
any pages which were touched will be synced for device.
Signed-off-by: Rebecca Schultz Zavin <rebecca@android.com>
[jstultz: modified patch to apply to staging directory] Signed-off-by: John Stultz <john.stultz@linaro.org> Signed-off-by: Greg Kroah-Hartman <gregkh@linuxfoundation.org>
projects / ~andy / linux / log