Calgary hits a NULL pointer dereference when booting in a multi-chassis
NUMA system. See Redhat bugzilla number 198498, found by Konrad
Rzeszutek ([email protected]).
There are many issues that had to be resolved to fix this problem.
Firstly when I originally wrote the code to handle NUMA systems, I
had a large misunderstanding that was not corrected until now. That was
that I thought the "number of nodes online" referred to number of
physical systems connected. So that if NUMA was disabled, there
would only be 1 node and it would only show that node's PCI bus.
In reality if NUMA is disabled, the system displays all of the
connected chassis as one node but is only ignorant of the delays
in accessing main memory. Therefore, references to num_online_nodes()
and MAX_NUMNODES are incorrect and need to be set to the maximum
number of nodes that can be accessed (which are 8). I created a
variable, MAX_NUM_CHASSIS, and set it to 8 to fix this.
Secondly, when walking the PCI in detect_calgary, the code only
checked the first "slot" when looking to see if a device is present.
This will work for most cases, but unfortunately it isn't always the
case. In the NUMA MXE drawers, there are USB devices present on the
3rd slot (with slot 1 being empty). So, to work around this, all
slots (up to 8) are scanned to see if there are any devices present.
Lastly, the bus is being enumerated on large systems in a different
way the we originally thought. This throws the ugly logic we had
out the window. To more elegantly handle this, I reorganized the
kva array to be sparse (which removed the need to have any bus number
to kva slot logic in tce.c) and created a secondary space array to
contain the bus number to phb mapping.
With these changes Calgary boots on an x460 with 4 nodes with and
without NUMA enabled.
Signed-off-by: Jon Mason <[email protected]>
Signed-off-by: Muli Ben-Yehuda <[email protected]>
Signed-off-by: Andi Kleen <[email protected]>
Signed-off-by: Linus Torvalds <[email protected]>
This patch hooks Calgary into the build, the x86-64 IOMMU
initialization paths, and introduces the Calgary specific bits. The
implementation draws inspiration from both PPC (which has support for
the same chip but requires firmware support which we don't have on
x86-64) and gart. Calgary is different from gart in that it support a
translation table per PHB, as opposed to the single gart aperture.
Changes from previous version:
* Addition of boot-time disablement for bus-level translation/isolation
(e.g, enable userspace DMA for things like X)
* Usage of newer IOMMU abstraction functions
Signed-off-by: Muli Ben-Yehuda <[email protected]>
Signed-off-by: Jon Mason <[email protected]>
Signed-off-by: Andi Kleen <[email protected]>
Signed-off-by: Linus Torvalds <[email protected]>
This patch creates a new interface for IOMMUs by adding a centralized
location for IOMMU allocation (for translation tables/apertures) and
IOMMU initialization. In creating these, code was moved around for
abstraction, uniformity, and consiceness.
Take note of the move of the iommu_setup bootarg parsing code to
__setup. This is enabled by moving back the location of the aperture
allocation/detection to mem init (which while ugly, was already the
location of the swiotlb_init).
While a slight departure from the previous patch, I belive this provides
the true intention of the previous versions of the patch which changed
this code. It also makes the addition of the upcoming calgary code much
cleaner than previous patches.
[AK: Removed one broken change. iommu_setup still has to be called
early]
Signed-off-by: Muli Ben-Yehuda <[email protected]>
Signed-off-by: Jon Mason <[email protected]>
Signed-off-by: Andi Kleen <[email protected]>
Signed-off-by: Linus Torvalds <[email protected]>
Based on Andi Kleen's comments on the original Calgary patch, move
valid_dma_direction into the calling functions.
Signed-off-by: Muli Ben-Yehuda <[email protected]>
Signed-off-by: Jon Mason <[email protected]>
Signed-off-by: Andi Kleen <[email protected]>
Signed-off-by: Linus Torvalds <[email protected]>
swiotlb relies on the gart specific iommu_aperture variable to know if
we discovered a hardware IOMMU before swiotlb initialization. Introduce
iommu_detected to do the same thing, but in a HW IOMMU neutral manner,
in preparation for adding the Calgary HW IOMMU.
Signed-Off-By: Muli Ben-Yehuda <[email protected]>
Signed-Off-By: Jon Mason <[email protected]>
Signed-off-by: Andi Kleen <[email protected]>
Signed-off-by: Linus Torvalds <[email protected]>
include/asm-x86_64/gart-mapping.h is only ever used in
arch/x86_64/kernel/setup.c and none of its contents are referenced.
Looks to be leftover cruft not removed in the dma_ops patch.
Signed-off-by: Jon Mason <[email protected]>
Signed-off-by: Andi Kleen <[email protected]>
Signed-off-by: Linus Torvalds <[email protected]>
A trivial change to have gart_unmap_sg call gart_unmap_single directly,
instead of bouncing through the dma_unmap_single wrapper in
dma-mapping.h.
This change required moving the gart_unmap_single above gart_unmap_sg,
and under gart_map_single (which seems a more logical place that its
current location IMHO).
Signed-off-by: Jon Mason <[email protected]>
Signed-off-by: Andi Kleen <[email protected]>
Signed-off-by: Linus Torvalds <[email protected]>
There already exists a big endian safe bitops implementation in
lib/find_next_bit.c. The code in it is 90%+ common with the powerpc
specific version, so the powerpc version is redundant. This patch
makes the necessary changes to use the generic bitops in powerpc, and
removes the powerpc specific version.
Signed-off-by: Jon Mason <[email protected]>
Signed-off-by: Paul Mackerras <[email protected]>
asm-ia64/bitops.h includes itself. The #ifndef _ASM_IA64_BITOPS_H
prevents this from being an issue, but it should still be removed.
Signed-off-by: Jon Mason <[email protected]>
Signed-off-by: Tony Luck <[email protected]>
Have the GART_IOMMU help text specify that this is the hardware IOMMU in
amd64 processors. This will be significant if/when other IOMMUs are
added to the x86-64 architecture. :-)
Also, note that the previous help text stated that IOMMU was needed for
>3GB memory instead of >4GB. This is fixed in the newer version.
Signed-off-by: Jon Mason <[email protected]>
Signed-off-by: Andi Kleen <[email protected]>
Signed-off-by: Linus Torvalds <[email protected]>
free_bootmem_node expects a physical address to be passed in, but
__alloc_bootmem_node returns a virtual one. That address needs to be
translated to physical.
Signed-off-by: Jon Mason <[email protected]>
Signed-off-by: Andi Kleen <[email protected]>
Signed-off-by: Linus Torvalds <[email protected]>
This patch fixes an error in the dl2k driver's DMA mapping/unmapping.
The adapter uses the upper 16bits of the DMA address for the buffer
size. However, this is not masked off when referencing the DMA
address, and can lead to errors by trying to free a DMA address out of
range.
Thanks,
Jon
Signed-off-by: Jon Mason <[email protected]>
Signed-off-by: Jeff Garzik <[email protected]>
In previous versions of pci-gart.c, no_iommu was used to determine if IOMMU was
disabled in the GART DMA mapping functions. This changed in 2.6.16 and now
gart_xxx() functions are only called if gart is enabled. Therefore, uses of
no_iommu in the GART code are no longer necessary and can be removed.
Also, it removes double deceleration of no_iommu and force_iommu in pci.h and
proto.h, by removing the deceleration in pci.h.
Lastly, end_pfn off by one error.
Tested (along with patch 1/2) on dual opteron with gart enabled, iommu=soft,
and iommu=off.
Signed-off-by: Jon Mason <[email protected]>
Signed-off-by: Andi Kleen <[email protected]>
Signed-off-by: Linus Torvalds <[email protected]>
This patch corrects a few spelling and grammar errors found in
drivers/net
Signed-off-by: Jon Mason <[email protected]>
Signed-off-by: John W. Linville <[email protected]>
This patch removes all self references and fixes references to files
in the now defunct arch/ppc64 tree. I think this accomplises
everything wanted, though there might be a few references I missed.
Signed-off-by: Jon Mason <[email protected]>
Signed-off-by: Paul Mackerras <[email protected]>
This patch contains a printk reorder to remove the current problem of
displaying "PCI-DMA: Disabling IOMMU." and then "PCI-DMA: using GART
IOMMU" 20 lines later in dmesg.
It also constains a printk reorder in swiotlb to state swiotlb
enablement prior to describing the location of the bounce buffers, and a
printk reorder to state gart enablement prior to describing the
aperature.
Also constains a whitespace cleanup in arch/x86_64/kernel/setup.c
Tested (along with patch 2/2) on dual opteron with gart enabled,
iommu=soft, and iommu=off.
Signed-off-by: Jon Mason <[email protected]>
Signed-off-by: Andi Kleen <[email protected]>
Signed-off-by: Linus Torvalds <[email protected]>
Modules: ALI5451 driver
This patch uses #defines for the Vendor ID and Device ID and uses the
new PCI_DEVICE macro.
Signed-off-by: Jon Mason <[email protected]>
Signed-off-by: Andrew Morton <[email protected]>
Signed-off-by: Takashi Iwai <[email protected]>
Modules: Trident driver
Some pcnet32 hardware erroneously has the Vendor ID for Trident. The
pcnet32 driver looks for the PCI ethernet class before grabbing the
hardware, but the current trident driver does not check against the
PCI audio class. This allows the trident driver to claim the pcnet32
hardware. This patch prevents that.
Per Jiri Slaby's request, I changed the trident driver to use
PCI_DEVICE macro and PCI ID #defines.
This patch is untested on Trident 4DWAVE_DX hardware, but has been
tested on pcnet32 hardware.
Signed-off-by: Jon Mason <[email protected]>
Signed-off-by: Takashi Iwai <[email protected]>
Some pcnet32 hardware erroneously has the Vendor ID for Trident. The
pcnet32 driver looks for the PCI ethernet class before grabbing the
hardware, but the current trident driver does not check against the PCI
audio class. This allows the trident driver to claim the pcnet32 hardware.
This patch prevents that.
This revised version of the OSS Trident patch includes PCI_DEVICE Macro
usage.
Signed-off-by: Jon Mason <[email protected]>
Signed-off-by: Muli Ben-Yehuda <[email protected]>
Signed-off-by: Andrew Morton <[email protected]>
Signed-off-by: Linus Torvalds <[email protected]>
Jon Mason