This is currently unused.
If a suspend must be limited to CPU level only by preventing the last man
from triggering a cluster level suspend then this should be determined
according to many other criteria the MCPM layer is currently not aware of.
It is unlikely that mcpm_cpu_suspend() would be the proper conduit for
that information anyway.
Signed-off-by: Nicolas Pitre <[email protected]>
Acked-by: Dave Martin <[email protected]>
This concerns the following helpers:
__mcpm_cpu_going_down()
__mcpm_cpu_down()
__mcpm_outbound_enter_critical()
__mcpm_outbound_leave_critical()
__mcpm_cluster_state()
They are and should only be used by the core code now. Therefore their
declarations are removed from mcpm.h and their definitions are made
static, hence the need to move them before their users which accounts
for the bulk of this patch.
This left the mcpm_sync_struct definition at an odd location, therefore
it is moved as well with some comment clarifications.
Signed-off-by: Nicolas Pitre <[email protected]>
Acked-by: Dave Martin <[email protected]>
Now that no one uses the old callbacks anymore, let's remove them
and associated support code.
Signed-off-by: Nicolas Pitre <[email protected]>
Acked-by: Dave Martin <[email protected]>
This platform is currently relying on the MCPM infrastructure for no
apparent reason. The MCPM concurrency handling brings no benefits here
as there is no asynchronous CPU wake-ups to be concerned about (this is
used for CPU hotplug and secondary boot only, not for CPU idle).
This platform is also different from the other MCPM users because a given
CPU can't shut itself down completely without the assistance of another
CPU. This is at odds with the on-going MCPM backend refactoring.
To simplify things, this is converted to hook directly into the
smp_operations callbacks, bypassing the MCPM infrastructure.
Tested-by: Wei Xu <[email protected]>
Cc: Haojian Zhuang <[email protected]>
Signed-off-by: Nicolas Pitre <[email protected]>
The custom suspend callback is removed for this change. The extra call
to exynos_cpu_power_up(() that was present at the end of exynos_suspend()
is now relocated to the cpu_is_up callback.
Signed-off-by: Nicolas Pitre <[email protected]>
Tested-by: Daniel Lezcano <[email protected]>
Signed-off-by: Olof Johansson <[email protected]>
Currently the cpu argument validity check uses a hardcoded limit of 4.
The DCSCB configuration data provides the actual number of CPUs and
we already use it elsewhere. Let's improve the cpu argument validity
check by using that information instead.
Signed-off-by: Nicolas Pitre <[email protected]>
Signed-off-by: Olof Johansson <[email protected]>
All backends are reimplementing a variation of the same CPU reference
count handling. They are also responsible for driving the MCPM special
low-level locking. This is needless duplication, involving algorithmic
requirements that are not necessarily obvious to the uninitiated.
And from past code review experience, those were all initially
implemented badly.
After 3 years, it is time to refactor as much common code to the core
MCPM facility to make the backends as simple as possible. To avoid a
flag day, the new scheme is introduced in parallel to the existing
backend interface. When all backends are converted over, the
compatibility interface could be removed.
The new MCPM backend interface implements simpler methods addressing
very platform specific tasks performed under lock protection while
keeping the algorithmic complexity and race avoidance local to the
core code.
Signed-off-by: Nicolas Pitre <[email protected]>
Tested-by: Daniel Lezcano <[email protected]>
Signed-off-by: Olof Johansson <[email protected]>
The vcs device's poll/fasync support relies on the vt notifier to signal
changes to the screen content. Notifier invocations were missing for
changes that comes through the selection interface though. Fix that.
Tested with BRLTTY 5.2.
Signed-off-by: Nicolas Pitre <[email protected]>
Cc: Dave Mielke <[email protected]>
Cc: stable <[email protected]>
Signed-off-by: Greg Kroah-Hartman <[email protected]>
There is currently a hardcoded limit of 64KB for the DTB to live in and
be extended with ATAG info. Some DTBs have outgrown that limit:
$ du -b arch/arm/boot/dts/omap3-n900.dtb
70212 arch/arm/boot/dts/omap3-n900.dtb
Furthermore, the actual size passed to atags_to_fdt() included the stack
size which is obviously wrong.
The initial DTB size is known, so use it to size the allocated workspace
with a 50% growth assumption and relocate the temporary stack above that.
This is also clamped to 32KB min / 1MB max for robustness against bad
DTB data.
Reported-by: Pali Rohár <[email protected]>
Tested-by: Pavel Machek <[email protected]>
Signed-off-by: Nicolas Pitre <[email protected]>
Signed-off-by: Russell King <[email protected]>
At least on ARM, do_div() is optimized to turn constant divisors into
an inline multiplication by the reciprocal value at compile time.
However this optimization is missed entirely whenever ktime_divns() is
used and the slow out-of-line division code is used all the time.
Let ktime_divns() use do_div() inline whenever the divisor is constant
and small enough. This will make things like ktime_to_us() and
ktime_to_ms() much faster.
Cc: Arnd Bergmann <[email protected]>
Cc: Thomas Gleixner <[email protected]>
Cc: Ingo Molnar <[email protected]>
Cc: Nicolas Pitre <[email protected]>
Acked-by: Arnd Bergmann <[email protected]>
Signed-off-by: Nicolas Pitre <[email protected]>
Signed-off-by: John Stultz <[email protected]>
This code was restored with commit 080fc66fb5 ("ARM: Bring back ARMv3 IO
and user access code") because the RiscPC memory bus does not understand
half-word load/stores. However only the IO code needed restoring since
the alternative user access code contains no half-word accesses, is
already used when CONFIG_PREEMPT is set and runs faster on a StrongARM.
Signed-off-by: Nicolas Pitre <[email protected]>
Signed-off-by: Russell King <[email protected]>
The code in find_idlest_cpu() looks for the CPU with the smallest load.
However, if multiple CPUs are idle, the first idle CPU is selected
irrespective of the depth of its idle state.
Among the idle CPUs we should pick the one with with the shallowest idle
state, or the latest to have gone idle if all idle CPUs are in the same
state. The later applies even when cpuidle is configured out.
This patch doesn't cover the following issues:
- The idle exit latency of a CPU might be larger than the time needed
to migrate the waking task to an already running CPU with sufficient
capacity, and therefore performance would benefit from task packing
in such case (in most cases task packing is about power saving).
- Some idle states have a non negligible and non abortable entry latency
which needs to run to completion before the exit latency can start.
A concurrent patch series is making this info available to the cpuidle
core. Once available, the entry latency with the idle timestamp could
determine when the exit latency may be effective.
Those issues will be handled in due course. In the mean time, what
is implemented here should improve things already compared to the current
state of affairs.
Based on an initial patch from Daniel Lezcano.
Signed-off-by: Nicolas Pitre <[email protected]>
Signed-off-by: Peter Zijlstra (Intel) <[email protected]>
Cc: Daniel Lezcano <[email protected]>
Cc: "Rafael J. Wysocki" <[email protected]>
Cc: Linus Torvalds <[email protected]>
Cc: [email protected]
Cc: [email protected]
Link: http://lkml.kernel.org/n/[email protected]
Signed-off-by: Ingo Molnar <[email protected]>
The strings used to list IPIs in /proc/interrupts are reused for tracing
purposes.
While at it, the code is slightly cleaned up so the ipi_types array
indices are no longer offset by IPI_RESCHEDULE whose value is 0 anyway.
Link: http://lkml.kernel.org/p/[email protected]
Acked-by: Will Deacon <[email protected]>
Acked-by: Catalin Marinas <[email protected]>
Signed-off-by: Nicolas Pitre <[email protected]>
Signed-off-by: Steven Rostedt <[email protected]>
The Inter Processor Interrupt is used to make another processor do a
specific action such as rescheduling tasks, signal a timer event or
execute something in another CPU's context. IRQs are already traceable
but IPIs were not. Tracing them is useful for monitoring IPI latency,
or to verify when they are the source of CPU wake-ups with power
management implications.
Three trace hooks are defined: ipi_raise, ipi_entry and ipi_exit. To make
them portable, a string is used to identify them and correlate related
events. Additionally, ipi_raise records a bitmask representing targeted
CPUs.
Link: http://lkml.kernel.org/p/[email protected]
Acked-by: Daniel Lezcano <[email protected]>
Signed-off-by: Nicolas Pitre <[email protected]>
Signed-off-by: Steven Rostedt <[email protected]>
With patch #8067/1 ("zImage: ensure header in LE format for BE8 kernels")
applied, it is no longer possible to determine the endianness of a compiled
kernel image. This normally shouldn't matter to the boot environment,
except for those cases where the selection of a ramdisk or root filesystem
with a matching endianness has to be automated.
Let's add a flag to the zImage header indicating the actual endianness.
Four bytes from offset 0x30 can be interpreted as follows:
04 03 02 01 big endian kernel
01 02 03 04 little endian kernel
Anything else should be interpreted as "unknown", in which case it is
most likely that patch #8067/1 was not applied either and the zImage
magic number at offset 0x24 could be used instead to determine
endianness. No zImage before this patch ever produced 0x01020304 nor
0x04030201 at offset 0x30 so there is no confusion possible.
Signed-off-by: Nicolas Pitre <[email protected]>
Acked-by: Kevin Hilman <[email protected]>
Signed-off-by: Russell King <[email protected]>
The Chromebook firmware doesn't enable the CCI for the boot cpu, and
arguably it shouldn't have to either. Let's have the kernel handle the
CCI on its own for the boot CPU the same way it does it for secondary CPUs
by using the MCPM loopback.
This allows to boot all 8 cores on exynos5420-peach-pit,
exynos5800-peach-pi and ARM Chromebook 2.
Signed-off-by: Nicolas Pitre <[email protected]>
Tested-by: Tushar Behera <[email protected]>
Reviewed-by: Kevin Hilman <[email protected]>
Tested-by: Kevin Hilman <[email protected]>
Tested-by: Doug Anderson <[email protected]>
Signed-off-by: Russell King <[email protected]>
This is not strictly needed on TC2 but still a good idea to exercise
that code.
Signed-off-by: nicolas Pitre <[email protected]>
Reviewed-by: Kevin Hilman <[email protected]>
Signed-off-by: Russell King <[email protected]>
The kernel already has the responsibility to handle resources such as the
CCI when hotplugging CPUs, during the booting of secondary CPUs, and when
resuming from suspend/idle. It would be more coherent and less confusing
if the CCI for the boot CPU (or cluster) was also initialized by the
kernel rather than expecting the firmware/bootloader to do it and only in
that case. After all, the kernel has all the necessary code already and
the bootloader shouldn't have to care at all.
The CCI may be turned on only when the cache is off. Leveraging the CPU
suspend code to loop back through the low-level MCPM entry point is all
that is needed to properly turn on the CCI from the kernel by using the
same code as during secondary boot.
Let's provide a generic MCPM loopback function that can be invoked by
backend initialization code to set things (CCI or similar) on the boot
CPU just as it is done for the other CPUs.
Signed-off-by: Nicolas Pitre <[email protected]>
Reviewed-by: Kevin Hilman <[email protected]>
Tested-by: Kevin Hilman <[email protected]>
Tested-by: Doug Anderson <[email protected]>
Signed-off-by: Russell King <[email protected]>
All known BE8-capable systems have LE bootloaders, so we need to ensure
that the magic number and image start/end values are in little endian
format.
[[email protected]: from nico's original email on this subject]
[[email protected]: removed lds.S->lds rule, added target to extra-y]
Signed-off-by: Nicolas Pitre <[email protected]>
Signed-off-by: Ben Dooks <[email protected]>
Signed-off-by: Taras Kondratiuk <[email protected]>
Signed-off-by: Russell King <[email protected]>
The sync_phys variable has been replaced by link time computation in
mcpm_head.S before the code was submitted upstream.
Signed-off-by: Nicolas Pitre <[email protected]>
Signed-off-by: Russell King <[email protected]>
It is better not to think about compute capacity as being equivalent
to "CPU power". The upcoming "power aware" scheduler work may create
confusion with the notion of energy consumption if "power" is used too
liberally.
This contains the architecture visible changes. Incidentally, only ARM
takes advantage of the available pow^H^H^Hcapacity scaling hooks and
therefore those changes outside kernel/sched/ are confined to one ARM
specific file. The default arch_scale_smt_power() hook is not overridden
by anyone.
Replacements are as follows:
arch_scale_freq_power --> arch_scale_freq_capacity
arch_scale_smt_power --> arch_scale_smt_capacity
SCHED_POWER_SCALE --> SCHED_CAPACITY_SCALE
SCHED_POWER_SHIFT --> SCHED_CAPACITY_SHIFT
The local usage of "power" in arch/arm/kernel/topology.c is also changed
to "capacity" as appropriate.
Signed-off-by: Nicolas Pitre <[email protected]>
Signed-off-by: Peter Zijlstra <[email protected]>
Cc: Vincent Guittot <[email protected]>
Cc: Daniel Lezcano <[email protected]>
Cc: Morten Rasmussen <[email protected]>
Cc: "Rafael J. Wysocki" <[email protected]>
Cc: [email protected]
Cc: Arnd Bergmann <[email protected]>
Cc: Dietmar Eggemann <[email protected]>
Cc: Grant Likely <[email protected]>
Cc: Linus Torvalds <[email protected]>
Cc: Mark Brown <[email protected]>
Cc: Rob Herring <[email protected]>
Cc: Russell King <[email protected]>
Cc: Sudeep KarkadaNagesha <[email protected]>
Cc: Vincent Guittot <[email protected]>
Cc: [email protected]
Cc: [email protected]
Cc: [email protected]
Link: http://lkml.kernel.org/n/[email protected]
Signed-off-by: Ingo Molnar <[email protected]>
It is better not to think about compute capacity as being equivalent
to "CPU power". The upcoming "power aware" scheduler work may create
confusion with the notion of energy consumption if "power" is used too
liberally.
Since struct sched_group_power is really about compute capacity of sched
groups, let's rename it to struct sched_group_capacity. Similarly sgp
becomes sgc. Related variables and functions dealing with groups are also
adjusted accordingly.
Signed-off-by: Nicolas Pitre <[email protected]>
Signed-off-by: Peter Zijlstra <[email protected]>
Cc: Vincent Guittot <[email protected]>
Cc: Daniel Lezcano <[email protected]>
Cc: Morten Rasmussen <[email protected]>
Cc: "Rafael J. Wysocki" <[email protected]>
Cc: [email protected]
Cc: Linus Torvalds <[email protected]>
Cc: [email protected]
Link: http://lkml.kernel.org/n/[email protected]
Signed-off-by: Ingo Molnar <[email protected]>
It is better not to think about compute capacity as being equivalent
to "CPU power". The upcoming "power aware" scheduler work may create
confusion with the notion of energy consumption if "power" is used too
liberally.
To make things explicit and not create more confusion with the existing
"capacity" member, let's rename things as follows:
power -> compute_capacity
capacity -> task_capacity
Note: none of those fields are actually used outside update_numa_stats().
Signed-off-by: Nicolas Pitre <[email protected]>
Signed-off-by: Peter Zijlstra <[email protected]>
Cc: Vincent Guittot <[email protected]>
Cc: Daniel Lezcano <[email protected]>
Cc: Morten Rasmussen <[email protected]>
Cc: "Rafael J. Wysocki" <[email protected]>
Cc: [email protected]
Cc: Linus Torvalds <[email protected]>
Cc: [email protected]
Link: http://lkml.kernel.org/n/[email protected]
Signed-off-by: Ingo Molnar <[email protected]>
The content of /sys/devices/system/cpu/cpu*/online is still 1 for those
CPUs that the switcher has removed even though the global state in
/sys/devices/system/cpu/online is updated correctly.
It turns out that commit 0902a9044f ("Driver core: Use generic
offline/online for CPU offline/online") has changed the way those files
retrieve their content by relying on on the generic attribute handling
code. The switcher, by calling cpu_down() directly, bypasses this
handling and the attribute value doesn't get updated.
Fix this by calling device_offline()/device_online() instead.
Signed-off-by: Nicolas Pitre <[email protected]>
Signed-off-by: Russell King <[email protected]>
It looks like the static mapping area for DMA was replaced by dynamic
allocation into the vmalloc area by commit e9da6e9905 but the
information in Documentation/arm/memory.txt was not removed accordingly.
CONSISTENT_END in arch/arm/include/asm/memory.h has no more users and
can be removed as well.
Signed-off-by: Nicolas Pitre <[email protected]>
Signed-off-by: Russell King <[email protected]>
The switcher should not depend on MAX_CLUSTER to determine ifit should
be activated or not. In a multiplatform kernel binary it is possible to
have dual-cluster and quad-cluster platforms configured in. In that case
MAX_CLUSTER which is a build time limit should be 4 and that shouldn't
prevent the switcher from working if the kernel is booted on a b.L
dual-cluster system.
In bL_switcher_halve_cpus() we already have a runtime validation check
to make sure we're dealing with only two clusters, so booting on a quad
cluster system will be caught and switcher activation aborted.
However, the b.L switcher must ensure the MCPM layer is initialized on
the booted hardware before doing anything. The mcpm_is_available()
function is added to that effect.
Signed-off-by: Nicolas Pitre <[email protected]>
Tested-by: Abhilash Kesavan <[email protected]>
Signed-off-by: Russell King <[email protected]>
Strictly speaking this call is a no-op on the platform where dcscb.c is
used since it only has architected caches. The call was there as a hint
to people inspired by this code when writing their own backend, but the
hint might not always be correct.
For example, if a PL310 were to be used it wouldn't be safe to call
the regular outer_flush_all() as atomic instructions for locking
are involved in that case and those instructions cannot be assumed to
still be operational after v7_exit_coherency_flush() has returned.
Given no other CPUs (in the cluster) should be running at that point
then standard concurrency concerns wouldn't apply.
So let's simply kill this call for now and enhance the existing comment.
Signed-off-by: Nicolas Pitre <[email protected]>
Signed-off-by: Russell King <[email protected]>
Integration of cpuidle with the scheduler requires that the idle loop be
closely integrated with the scheduler proper. Moving cpu/idle.c into the
sched directory will allow for a smoother integration, and eliminate a
subdirectory which contained only one source file.
Signed-off-by: Nicolas Pitre <[email protected]>
Signed-off-by: Peter Zijlstra <[email protected]>
Cc: Linus Torvalds <[email protected]>
Cc: Andrew Morton <[email protected]>
Cc: Thomas Gleixner <[email protected]>
Link: http://lkml.kernel.org/r/[email protected]
Signed-off-by: Ingo Molnar <[email protected]>
The core idle loop now takes care of it. However a few things need
checking:
- Invocation of cpuidle_idle_call() in pseries_lpar_idle() happened
through arch_cpu_idle() and was therefore always preceded by a call
to ppc64_runlatch_off(). To preserve this property now that
cpuidle_idle_call() is invoked directly from core code, a call to
ppc64_runlatch_off() has been added to idle_loop_prolog() in
platforms/pseries/processor_idle.c.
- Similarly, cpuidle_idle_call() was followed by ppc64_runlatch_off()
so a call to the later has been added to idle_loop_epilog().
- And since arch_cpu_idle() always made sure to re-enable IRQs if they
were not enabled, this is now
done in idle_loop_epilog() as well.
The above was made in order to keep the execution flow close to the
original. I don't know if that was strictly necessary. Someone well
aquainted with the platform details might find some room for possible
optimizations.
Signed-off-by: Nicolas Pitre <[email protected]>
Reviewed-by: Preeti U Murthy <[email protected]>
Cc: "Rafael J. Wysocki" <[email protected]>
Cc: Daniel Lezcano <[email protected]>
Cc: [email protected]
Cc: [email protected]
Cc: [email protected]
Cc: [email protected]
Cc: Russell King <[email protected]>
Cc: [email protected]
Cc: Benjamin Herrenschmidt <[email protected]>
Cc: Paul Mundt <[email protected]>
Signed-off-by: Peter Zijlstra <[email protected]>
Link: http://lkml.kernel.org/n/[email protected]
Signed-off-by: Ingo Molnar <[email protected]>
In order to integrate cpuidle with the scheduler, we must have a better
proximity in the core code with what cpuidle is doing and not delegate
such interaction to arch code.
Architectures implementing arch_cpu_idle() should simply enter
a cheap idle mode in the absence of a proper cpuidle driver.
In both cases i.e. whether it is a cpuidle driver or the default
arch_cpu_idle(), the calling convention expects IRQs to be disabled
on entry and enabled on exit. There is a warning in place already but
let's add a forced IRQ enable here as well. This will allow for
removing the forced IRQ enable some implementations do locally and
allowing for the warning to trig.
Signed-off-by: Nicolas Pitre <[email protected]>
Acked-by: Daniel Lezcano <[email protected]>
Cc: Benjamin Herrenschmidt <[email protected]>
Cc: Preeti U Murthy <[email protected]>
Cc: Paul Mundt <[email protected]>
Cc: "Rafael J. Wysocki" <[email protected]>
Cc: Olof Johansson <[email protected]>
Cc: Russell King <[email protected]>
Cc: Linus Torvalds <[email protected]>
Signed-off-by: Peter Zijlstra <[email protected]>
Link: http://lkml.kernel.org/r/[email protected]
Signed-off-by: Ingo Molnar <[email protected]>
Now that we select HAVE_EFFICIENT_UNALIGNED_ACCESS for ARMv6+ CPUs,
replace the __LINUX_ARM_ARCH__ check in uaccess.h with the new symbol.
Signed-off-by: Nicolas Pitre <[email protected]>
Signed-off-by: Will Deacon <[email protected]>
Signed-off-by: Russell King <[email protected]>
So any FIQ handling is superfluous at the moment. The functions to
disable/enable FIQs is kept around if ever someone needs them in the
future, but existing calling sites including arch_cpu_idle_prepare()
may go for now.
Signed-off-by: Nicolas Pitre <[email protected]>
Signed-off-by: Catalin Marinas <[email protected]>
We have a handy macro to replace open coded __cpuc_flush_dcache_area(()
and outer_clean_range() sequences. Let's use it. No functional change.
Signed-off-by: Nicolas Pitre <[email protected]>
Signed-off-by: Russell King <[email protected]>
We have a handy macro to replace open coded __cpuc_flush_dcache_area(()
and outer_clean_range() sequences. Let's use it. No functional change.
Signed-off-by: Nicolas Pitre <[email protected]>
Signed-off-by: Olof Johansson <[email protected]>
The b.L switcher can be turned on/off at run time. It is therefore
necessary to change the cpufreq driver behavior accordingly.
The driver must be unregistered/registered with the cpufreq core
to reconfigure freq tables for the virtual or actual CPUs. This is
accomplished via the b.L switcher notifier callback.
Signed-off-by: Nicolas Pitre <[email protected]>
Signed-off-by: Viresh Kumar <[email protected]>
Signed-off-by: Rafael J. Wysocki <[email protected]>
This code is becoming duplicated in many places. So let's consolidate
it into a handy macro that is known to be right and available for reuse.
Signed-off-by: Nicolas Pitre <[email protected]>
Acked-by: Dave Martin <[email protected]>
Signed-off-by: Russell King <[email protected]>
Currently mcpm_cpu_power_down() and mcpm_cpu_suspend() trigger BUG()
if mcpm_platform_register() is not called beforehand. This may occur
for many reasons such as some incomplete device tree passed to the kernel
or the like.
Let's be nicer to users and avoid killing the kernel if that happens by
logging a warning and returning to the caller. The mcpm_cpu_suspend()
user is already set to deal with this situation, and so is cpu_die()
invoking mcpm_cpu_die().
The problematic case would have been the B.L switcher's usage of
mcpm_cpu_power_down(), however it has to call mcpm_cpu_power_up() first
which is already set to catch an error resulting from a missing
mcpm_platform_register() call.
Signed-off-by: Nicolas Pitre <[email protected]>
Signed-off-by: Russell King <[email protected]>
In some cases, a significant delay may be observed between the moment
a request for a CPU to come up is made and the moment it is ready to
start executing kernel code. This is especially true when a whole
cluster has to be powered up which may take in the order of miliseconds.
It is therefore a good idea to let the outbound CPU continue to execute
code in the mean time, and be notified when the inbound is ready before
performing the actual switch.
This is achieved by registering a completion block with the appropriate
IPI callback, and programming the sending of an IPI by the early assembly
code prior to entering the main kernel code. Once the IPI is delivered
to the outbound CPU, the completion block is "completed" and the switcher
thread is resumed.
Signed-off-by: Nicolas Pitre <[email protected]>
The regular gic_raise_softirq() takes as input a CPU mask which is not
adequate when we need to send an IPI to a CPU which is not represented
in the kernel to GIC mapping. That is the case with the b.L switcher
when GIC migration to the inbound CPU has not yet occurred.
Signed-off-by: Nicolas Pitre <[email protected]>
In order to have early assembly code signal other CPUs in the system,
we need to get the physical address for the SGIR register used to
send IPIs. Because the register will be used with a precomputed CPU
interface ID number, there is no need for any locking in the assembly
code where this register is written to.
Signed-off-by: Nicolas Pitre <[email protected]>
This allows to poke a predetermined value into a specific address
upon entering the early boot code in bL_head.S.
Signed-off-by: Nicolas Pitre <[email protected]>
We need a mechanism to let an inbound CPU signal that it is alive before
even getting into the kernel environment i.e. from early assembly code.
Using an IPI is the simplest way to achieve that.
This adds some basic infrastructure to register a struct completion
pointer to be "completed" when the dedicated IPI for this task is
received.
Signed-off-by: Nicolas Pitre <[email protected]>
Let's wait for the inbound CPU to come up and snoop some of the outbound
CPU cache before bringing the outbound CPU down. That should be more
efficient than going down right away.
Possible improvements might involve some monitoring of the CCI event
counters.
Signed-off-by: Nicolas Pitre <[email protected]>
This fixes the following build error:
/tmp/cce439dZ.s: Assembler messages:
/tmp/cce439dZ.s:506: Error: selected processor does not support ARM mode `isb '
/tmp/cce439dZ.s:512: Error: selected processor does not support ARM mode `isb '
/tmp/cce439dZ.s:513: Error: selected processor does not support ARM mode `dsb '
/tmp/cce439dZ.s:583: Error: selected processor does not support ARM mode `isb '
/tmp/cce439dZ.s:589: Error: selected processor does not support ARM mode `isb '
/tmp/cce439dZ.s:590: Error: selected processor does not support ARM mode `dsb '
Tested-by: Guenter Roeck <[email protected]>
Signed-off-by: Nicolas Pitre <[email protected]>
Tested-by: Kevin Hilman <[email protected]>
Signed-off-by: Olof Johansson <[email protected]>
When processors are about to hit low power states, the assertion of
standbywfi signal, triggered by the wfi instruction, is essential to
entering low power modes. If an IRQ is pending on the processor at the
time wfi is issued, the wfi instruction completes and the processor
restarts execution without asserting the standbywfi signal. Depending
on the platform power controller HW this behaviour can be acceptable or
not; if this behaviour must be prevented software should be provided
with a way to disable the routing of interrupts to the core IRQ pins.
On systems where raw GIC distributor interrupts are connected to the power
controller as wake-up events (hence the power controller still senses
IRQs and can wake up cores upon IRQ pending), the GIC CPU interface can
be disabled on power down, so that the GIC CPU IF output is gated and wfi
cannot complete, thereby preventing the standbywfi issue.
This patch adds a simple function to the GIC driver that allows to
disable the GIC CPU IF from power down procedures.
Signed-off-by: Nicolas Pitre <[email protected]>
Signed-off-by: Lorenzo Pieralisi <[email protected]>
[rewrote commit log]
Signed-off-by: Olof Johansson <[email protected]>
Commit f6f91b0d9f ("ARM: allow kuser helpers to be removed from the
vector page") introduced some help text for the CONFIG_KUSER_HELPERS
option which is rather contradictory.
Let's fix that, and improve it a little.
Cc: <[email protected]>
Signed-off-by: Nicolas Pitre <[email protected]>
Signed-off-by: Russell King <[email protected]>
If CONFIG_FRAME_POINTER=y we get the following error:
arch/arm/mach-vexpress/tc2_pm.c: In function 'tc2_pm_down':
arch/arm/mach-vexpress/tc2_pm.c:200:1: error: fp cannot be used in asm here
Let's fix that by explicitly preserving r11 on the stack and removing it
from the clobber list.
Reported-by: Russell King <[email protected]>
Reviewed-by: Dave Martin <[email protected]>
Signed-off-by: Nicolas Pitre <[email protected]>
Signed-off-by: Olof Johansson <[email protected]>
Similar to power_down(), except that for a suspend, the firmware
mailbox address has to be set prior entering low power mode.
The residency argument is not used yet, so the last man always shuts
down the cluster for now.
Signed-off-by: Nicolas Pitre <[email protected]>
Acked-by: Pawel Moll <[email protected]>
This is the MCPM backend for the Virtual Express A15x2 A7x3 CoreTile
aka TC2. This provides cluster management for SMP secondary boot and
CPU hotplug.
Signed-off-by: Nicolas Pitre <[email protected]>
Acked-by: Pawel Moll <[email protected]>
Reviewed-by: Lorenzo Pieralisi <[email protected]>
[PM: made it drive SCC registers directly and provide base for SPC]
Signed-off-by: Pawel Moll <[email protected]>
Only the basic call to aid debugging.
*** NOT FOR PRODUCTION ***
Usage:
echo <cpuid>,<clusterid> > /dev/b.L_switcher
where <cpuid> is the logical CPU number, and <clusterid> is 0 for the
first cluster and 1 for the second cluster.
Signed-off-by: nicolas Pitre <[email protected]>
Trying to support both the switcher and CPU hotplug at the same time
is tricky due to ambiguous semantics. So let's at least prevent users
from messing around with those logical CPUs the switcher has removed
and those which were not active when the switcher was activated.
Signed-off-by: Nicolas Pitre <[email protected]>
Up to now, the logical CPU was somehow tied to the physical CPU number
within a cluster. This causes problems when forcing the boot CPU to be
different from the first enumerated CPU in the device tree creating a
discrepancy between logical and physical CPU numbers.
Let's make the pairing completely independent from physical CPU numbers.
Let's keep only those logical CPUs with same initial CPU cluster to create
a uniform scheduler profile without having to modify any of the probed
topology and compute capacity data. This has the potential to create
a non contiguous CPU numbering space when the switcher is active with
potential impact on buggy user space tools. It is however better to fix
those tools rather than making the switcher code more intrusive.
Signed-off-by: Nicolas Pitre <[email protected]>
Reviewed-by: Lorenzo Pieralisi <[email protected]>
By adding no_bL_switcher to the kernel cmdline string, the switcher
won't be activated automatically at boot time. It is still possible
to activate it later with:
echo 1 > /sys/kernel/bL_switcher/active
Signed-off-by: Nicolas Pitre <[email protected]>
The /sys/kernel/bL_switcher/enable file allows to enable or disable
the switcher by writing 1 or 0 to it respectively. It is still enabled
by default on boot.
Signed-off-by: Nicolas Pitre <[email protected]>
Currently, GIC IDs are hardcoded making the code dependent on the 4+4 b.L
configuration. Let's allow for GIC IDs to be discovered upon switcher
initialization to support other b.L configurations such as the 1+1 one,
or 2+3 as on the VExpress TC2.
Signed-off-by: Nicolas Pitre <[email protected]>
In a regular kernel configuration, all the CPUs are initially available.
But the switcher execution model uses half of them at any time. Instead
of hacking the DTB to remove half of the CPUs, let's remove them at
run time and make sure we still have a working switcher configuration.
This way, the same DTB can be used whether or not the switcher is used.
Signed-off-by: Nicolas Pitre <[email protected]>
The workqueues are problematic as they may be contended.
They can't be scheduled with top priority either. Also the optimization
in bL_switch_request() to skip the workqueue entirely when the target CPU
and the calling CPU were the same didn't allow for bL_switch_request() to
be called from atomic context, as might be the case for some cpufreq
drivers.
Let's move to dedicated kthreads instead.
Signed-off-by: Nicolas Pitre <[email protected]>
This is the core code implementing big.LITTLE switcher functionality.
Rationale for this code is available here:
http://lwn.net/Articles/481055/
The main entry point for a switch request is:
void bL_switch_request(unsigned int cpu, unsigned int new_cluster_id)
If the calling CPU is not the wanted one, this wrapper takes care of
sending the request to the appropriate CPU with schedule_work_on().
At the moment the core switch operation is handled by bL_switch_to()
which must be called on the CPU for which a switch is requested.
What this code does:
* Return early if the current cluster is the wanted one.
* Close the gate in the kernel entry vector for both the inbound
and outbound CPUs.
* Wake up the inbound CPU so it can perform its reset sequence in
parallel up to the kernel entry vector gate.
* Migrate all interrupts in the GIC targeting the outbound CPU
interface to the inbound CPU interface, including SGIs. This is
performed by gic_migrate_target() in drivers/irqchip/irq-gic.c.
* Call cpu_pm_enter() which takes care of flushing the VFP state to
RAM and save the CPU interface config from the GIC to RAM.
* Modify the cpu_logical_map to refer to the inbound physical CPU.
* Call cpu_suspend() which saves the CPU state (general purpose
registers, page table address) onto the stack and store the
resulting stack pointer in an array indexed by the updated
cpu_logical_map, then call the provided shutdown function.
This happens in arch/arm/kernel/sleep.S.
At this point, the provided shutdown function executed by the outbound
CPU ungates the inbound CPU. Therefore the inbound CPU:
* Picks up the saved stack pointer in the array indexed by its MPIDR
in arch/arm/kernel/sleep.S.
* The MMU and caches are re-enabled using the saved state on the
provided stack, just like if this was a resume operation from a
suspended state.
* Then cpu_suspend() returns, although this is on the inbound CPU
rather than the outbound CPU which called it initially.
* The function cpu_pm_exit() is called which effect is to restore the
CPU interface state in the GIC using the state previously saved by
the outbound CPU.
* Exit of bL_switch_to() to resume normal kernel execution on the
new CPU.
However, the outbound CPU is potentially still running in parallel while
the inbound CPU is resuming normal kernel execution, hence we need
per CPU stack isolation to execute bL_do_switch(). After the outbound
CPU has ungated the inbound CPU, it calls mcpm_cpu_power_down() to:
* Clean its L1 cache.
* If it is the last CPU still alive in its cluster (last man standing),
it also cleans its L2 cache and disables cache snooping from the other
cluster.
* Power down the CPU (or whole cluster).
Code called from bL_do_switch() might end up referencing 'current' for
some reasons. However, 'current' is derived from the stack pointer.
With any arbitrary stack, the returned value for 'current' and any
dereferenced values through it are just random garbage which may lead to
segmentation faults.
The active page table during the execution of bL_do_switch() is also a
problem. There is no guarantee that the inbound CPU won't destroy the
corresponding task which would free the attached page table while the
outbound CPU is still running and relying on it.
To solve both issues, we borrow some of the task space belonging to
the init/idle task which, by its nature, is lightly used and therefore
is unlikely to clash with our usage. The init task is also never going
away.
Right now the logical CPU number is assumed to be equivalent to the
physical CPU number within each cluster. The kernel should also be
booted with only one cluster active. These limitations will be lifted
eventually.
Signed-off-by: Nicolas Pitre <[email protected]>
This is required by the big.LITTLE switcher code.
The gic_migrate_target() changes the CPU interface mapping for the
current CPU to redirect SGIs to the specified interface, and it also
updates the target CPU for each interrupts to that CPU interface
if they were targeting the current interface. Finally, pending
SGIs for the current CPU are forwarded to the new interface.
Because Linux does not use it, the SGI source information for the
forwarded SGIs is not preserved. Neither is the source information
for the SGIs sent by the current CPU to other CPUs adjusted to match
the new CPU interface mapping. The required registers are banked so
only the target CPU could do it.
Signed-off-by: Nicolas Pitre <[email protected]>
Currently we hash the MPIDR of the CPU being suspended to determine which
entry in the sleep_save_sp array to use. In some situations, such as when
we want to resume on another physical CPU, the MPIDR of another CPU should
be used instead.
So let's use the value of cpu_logical_map(smp_processor_id()) in place
of the MPIDR in the suspend path. This will result in the same index
being used as with the previous code unless the caller has modified
cpu_logical_map() beforehand with the MPIDR of the physical CPU the
suspending logical CPU will resume on.
Consequently, if doing a physical CPU migration, cpu_logical_map() must
be updated appropriately somewhere between cpu_pm_enter() and
cpu_suspend().
The register allocation in __cpu_suspend is reworked in order to better
accommodate the additional argument.
Signed-off-by: Nicolas Pitre <[email protected]>
Acked-by: Lorenzo Pieralisi <[email protected]>
Reviewed-by: Dave Martin <[email protected]>
Unlike real A15/A7's, the RTSM simulation doesn't appear to hit the
cache when the CTRL.C bit is cleared. Let's ensure there is no memory
access within the disable and flush cache sequence, including to the
stack.
Signed-off-by: Nicolas Pitre <[email protected]>
When zImage is loaded into RAM at a low address but TEXT_OFFSET
is set higher, we risk overwriting ourself with the page table
needed to turn on the cache as it is located relative to the relocation
address. Let's defer the cache setup after relocation in that case.
Signed-off-by: Nicolas Pitre <[email protected]>
Reported-by: Stephen Boyd <[email protected]>
Tested-by: Stephen Boyd <[email protected]>
Signed-off-by: Russell King <[email protected]>
The smp_init_cpus method in the smp_operations structure is optional
and can be omitted entirely.
Signed-off-by: Nicolas Pitre <[email protected]>
Signed-off-by: Russell King <[email protected]>
The CPU and cluster numbers (MPIDR affinity levels 0 and 1) may have
at most 8 bits each.
Signed-off-by: Nicolas Pitre <[email protected]>
Signed-off-by: Russell King <[email protected]>
Selecting this option produces:
AS arch/arm/boot/compressed/debug.o
arch/arm/boot/compressed/debug.S:4:33: fatal error: mach/debug-macro.S: No such file or directory
compilation terminated.
make[3]: *** [arch/arm/boot/compressed/debug.o] Error 1
The semihosting support cannot be modelled into a senduart macro as
it requires memory space for argument passing. So the
CONFIG_DEBUG_LL_INCLUDE may not have any sensible value and the include
directive should be omitted.
While at it, let's add proper semihosting output support to the
decompressor.
Signed-off-by: Nicolas Pitre <[email protected]>
Acked-by: Shawn Guo <[email protected]>
Signed-off-by: Russell King <[email protected]>
If 4 CPUs are assumed, the A15x1-A7x1 model configuration would never
shut down the initial cluster as the 0xf reset bit mask will never be
observed. Let's construct this mask based on the provided information
in the DCSCB config register for the number of CPUs per cluster.
Signed-off-by: Nicolas Pitre <[email protected]>
Reviewed-by: Santosh Shilimkar <[email protected]>
Acked-by: Pawel Moll <[email protected]>
It is possible for a CPU to be told to power up before it managed
to power itself down. Solve this race with a usage count to deal
with this possibility as mandated by the MCPM API definition.
Signed-off-by: nicolas Pitre <[email protected]>
Reviewed-by: Santosh Shilimkar <[email protected]>
Acked-by: Pawel Moll <[email protected]>
This adds basic CPU and cluster reset controls on RTSM for the
A15x4-A7x4 model configuration using the Dual Cluster System
Configuration Block (DCSCB).
The cache coherency interconnect (CCI) is not handled yet.
Signed-off-by: Nicolas Pitre <[email protected]>
Reviewed-by: Santosh Shilimkar <[email protected]>
Acked-by: Pawel Moll <[email protected]>
This provides cci_enable_port_for_self(). This is the counterpart to
cci_disable_port_by_cpu(self).
This is meant to be called from the MCPM machine specific power_up_setup
callback code when the appropriate affinity level needs to be initialized.
The code therefore has to be position independent as the MMU is still off
and it cannot rely on any stack space.
Signed-off-by: Nicolas Pitre <[email protected]>
Reviewed-by: Dave Martin <[email protected]>
Since commit c0114709ed ("irqchip: gic: Perform the gic_secondary_init()
call via CPU notifier") it is no longer required nor possible to call
gic_secondary_init() from platform code.
Signed-off-by: Nicolas Pitre <[email protected]>
Signed-off-by: Russell King <[email protected]>
Now that the cluster power API is in place, we can use it for SMP secondary
bringup and CPU hotplug in a generic fashion.
Signed-off-by: Nicolas Pitre <[email protected]>
Reviewed-by: Santosh Shilimkar <[email protected]>
Reviewed-by: Will Deacon <[email protected]>
This is the basic API used to handle the powering up/down of individual
CPUs in a (multi-)cluster system. The platform specific backend
implementation has the responsibility to also handle the cluster level
power as well when the first/last CPU in a cluster is brought up/down.
Signed-off-by: Nicolas Pitre <[email protected]>
Reviewed-by: Santosh Shilimkar <[email protected]>
Reviewed-by: Will Deacon <[email protected]>
CPUs in cluster based systems, such as big.LITTLE, have special needs
when entering the kernel due to a hotplug event, or when resuming from
a deep sleep mode.
This is vectorized so multiple CPUs can enter the kernel in parallel
without serialization.
The mcpm prefix stands for "multi cluster power management", however
this is usable on single cluster systems as well. Only the basic
structure is introduced here. This will be extended with later patches.
In order not to complexify things more than they currently have to,
the planned work to make runtime adjusted MPIDR based indexing and
dynamic memory allocation for cluster states is postponed to a later
cycle. The MAX_NR_CLUSTERS and MAX_CPUS_PER_CLUSTER static definitions
should be sufficient for those systems expected to be available in the
near future.
Signed-off-by: Nicolas Pitre <[email protected]>
Reviewed-by: Santosh Shilimkar <[email protected]>
Reviewed-by: Will Deacon <[email protected]>
Algorithms used by the MCPM layer rely on state variables which are
accessed while the cache is either active or inactive, depending
on the code path and the active state.
This patch introduces generic cache maintenance helpers to provide the
necessary cache synchronization for such state variables to always hit
main memory in an ordered way.
Signed-off-by: Nicolas Pitre <[email protected]>
Acked-by: Russell King <[email protected]>
Acked-by: Dave Martin <[email protected]>
Commit 455bd4c430 ("ARM: 7668/1: fix memset-related crashes caused by
recent GCC (4.7.2) optimizations") attempted to fix a compliance issue
with the memset return value. However the memset itself became broken
by that patch for misaligned pointers.
This fixes the above by branching over the entry code from the
misaligned fixup code to avoid reloading the original pointer.
Also, because the function entry alignment is wrong in the Thumb mode
compilation, that fixup code is moved to the end.
While at it, the entry instructions are slightly reworked to help dual
issue pipelines.
Signed-off-by: Nicolas Pitre <[email protected]>
Tested-by: Alexander Holler <[email protected]>
Signed-off-by: Russell King <[email protected]>
Building the kernel with allyesconfig fails because the i.mx early
resume code located in the .data section is unable to fixup the bl
relocation as the branch target gets too far away.
The idea of having code in the .data section allows for easy access to
nearby data using relative addressing while the MMU is off. However it
is probably best to move the code back to the .text section where it
belongs and fixup the data access instead. This solves the bl reloc
issue (at least until this becomes a general problem) and simplifies
the code as well.
Signed-off-by: Nicolas Pitre <[email protected]>
Signed-off-by: Shawn Guo <[email protected]>
Commit 81732c3b2f ("tty vt: Fix line garbage in virtual console on
command line edition") broke insert_char() in multiple ways. Then
commit b1a925f44a ("tty vt: Fix a regression in command line edition")
partially fixed it. However, the buffer being moved is still too large
and overflowing beyond the end of the current line, corrupting existing
characters on the next line.
Example test case:
echo -e "abc\nde\x1b[A\x1b[4h \x1b[4l\x1b[B"
Expected result:
ab c
de
Current result:
ab c
e
Needless to say that this is very annoying when inserting words in the
middle of paragraphs with certain text editors.
Signed-off-by: Nicolas Pitre <[email protected]>
Cc: Jean-François Moine <[email protected]>
Cc: Greg Kroah-Hartman <[email protected]>
Cc: <[email protected]>
Signed-off-by: Linus Torvalds <[email protected]>
When udelay() is implemented using an architected timer, it is wrong
to scale loops_per_jiffy when changing the CPU clock frequency since
the timer clock remains constant.
The lpj should probably become an implementation detail relevant to
the CPU loop based delay routine only and more confined to it. In the
mean time this is the minimal fix needed to have expected delays with
the timer based implementation when cpufreq is also in use.
Reported-by: Viresh Kumar <[email protected]>
Signed-off-by: Nicolas Pitre <[email protected]>
Tested-by: Viresh Kumar <[email protected]>
Acked-by: Liviu Dudau <[email protected]>
Cc: [email protected]
Signed-off-by: Russell King <[email protected]>
We currently use a temporary 1MB section aligned to a 1MB boundary for
mapping the provided device tree until the final page table is created.
However, if the device tree happens to cross that 1MB boundary, the end
of it remains unmapped and the kernel crashes when it attempts to access
it. Given no restriction on the location of that DTB, it could end up
with only a few bytes mapped at the end of a section.
Solve this issue by mapping two consecutive sections.
Signed-off-by: Nicolas Pitre <[email protected]>
Tested-by: Sascha Hauer <[email protected]>
Tested-by: Tomasz Figa <[email protected]>
Cc: [email protected]
Signed-off-by: Russell King <[email protected]>
This is what is done for the regular interrupts in kernel/irqs/proc.c
already, before calling arch_show_interrupts(). Not doing so for the
IPIs causes the column headers not to match with the content whenever
some CPUs are offline.
Signed-off-by: Nicolas Pitre <[email protected]>
Signed-off-by: Russell King <[email protected]>
The kvm_seq value has nothing to do what so ever with this other KVM.
Given that KVM support on ARM is imminent, it's best to rename kvm_seq
into something else to clearly identify what it is about i.e. a sequence
number for vmalloc section mappings.
Signed-off-by: Nicolas Pitre <[email protected]>
Signed-off-by: Russell King <[email protected]>
The GIC interface numbering does not necessarily follow the logical
CPU numbering, especially for complex topologies such as multi-cluster
systems.
Fortunately we can easily probe the GIC to create a mapping as the
Interrupt Processor Targets Registers for the first 32 interrupts are
read-only, and each field returns a value that always corresponds to
the processor reading the register.
Initially all mappings target all CPUs in case an IPI is required to
boot secondary CPUs. It is refined as those CPUs discover what their
actual mapping is.
Signed-off-by: Nicolas Pitre <[email protected]>
Acked-by: Will Deacon <[email protected]>
Nicolas Pitre