Most of this information has been what I have personally collected and/or told from a reliable source.
The first thing you must accept is the base line.....
The RS4 engine in the engine bay will not make the full 414bhp standard.
No doubt when Audi tested the engine, it would have been on a dyno rig with no pumps and possibly a different exhaust and intake airfilter.
So my base line for a healthy car is 370-390hp.
Golden rule number one, if you are going to have a remap, make sure you car is healthy. On a RR you should see the same shape as what has been published. If you see anything else your car has a problem and needs to be investigated. If you are given an amazing number after a remap check the shape and make sure you see the at wheel graphs. A Dyno operator can apply the handbrake when doing the 'coastdown', thus the Dyno software could see a greater transmission drag number .. so the at engine number will loook fab. No standard RS4 with a remap will ever exceed 400hp. If it does take it somewhere else spend the £35 and be prepared to be disappointed.
Fitting an ITG/K&N does not harm the power output of the car .. probably buys you 1-3hp.
Fitting a miltek or other after market catback exhaust will cost you power with a factory map(up to 0080, don't know about 0090 yet). Why, well it seems this engine is very sensitive to changes and the ECU holds things back.
So we now have clue number one ... the ECU is sensitive to changes ...
So lets move onto the carbon build up. Audi know all about this. It is a fundamental fact that Direct Injection engines produce carbon. Hence why the Wynn's product has been developed.
The reality is that performance is not damaged by the build up. How can I say this? RAudiguy's(never cleaned) car performs very close to Pippyrips car, which had been fully cleaned. Both run custom maps and have no cats ... they are the will above 400Hp boys ...
The carbon deposits can potentially causes a problems longterm or in some cases sooner. Silver's car had an injector problem more than likely a damaged injector. This also brings me to the point that Super unleaded petrol should always be used to help remove anything on the 'wet side' but obviously to allow the ECU to advance the ignition as much as possible. There is logic to say it is worth while filling an empty tank with BP102(£2.42per Litre) and then going for a blat at Brands Hatch. Not only does the fuel have excellent cleaning properties the ECU should really advance the ignition due to the golden nectar

So now we have a picture that this engine is complicated and very much an electronic beast. As the RPM increases and depending if in Sport mode or not, lots of things open. We have the tail pipe exhaust valves(these have been found to be stuck), we have the flapper valve in the airbox and also the tumble vanes in the inlet manifold. All of these items are operated by vacuum and controlled by the ECU.
So we have two potential issues either a pipe could be damaged/lever jammed or indeed the ECU NOT seeing what it expects so holds things back. Again the ECU is relying on input from sensors to see the correct environment for power to take off.
We have all heard of MAF's being replaced and also other sensors. some live in a very harsh environment. To me I would have thought either a sensor works or does not .. but clearly we have seen Oli's car with a duff MAF(fault shown) but the car still appear to run ok on the road....but down on power.
On the oil front, it has been proven that the engine beats the crap out of it. I changed mine at 2.5k miles and almost every 5k thereafter. This might be the reason my car dynoed well at SRR. Regular new oil will be less prone to the carbon deposits and will be a 'lighter 'vapour in the inlet.
For the cars not on the button, I have no doubt that the engines are mechanically sound/good, but have an external item/pipe broken or a duff sensor.
As far as mods go, there are now lots of options depending on your pocket, but you must start with a healthy car.