Given that the Nomad with 113 psi in the intake and a 12:1 ratio diesel produced 3500 hp and 750 lbs static thrust on a 1200 cu in engine with turbo-compounding, I'd say 2.7 hp/cu. in. with normal fuels (no nitrous oxide, etc) is about the limit. About 20% of that 3500 hp came from having the turbo drive the crankshaft. Eight stage axial compressor and three stage axial turbine -- RR added three feet to the shaft and put compustion chambers between the front and rear instead of a piston engine and produced something like 8000 lbs static thrust (aka the Rolls Avon turbojet, eventually upgraded to the Rolls Conway 1:1 bypass turbofan by extending the rear set of turbine blade to make a fan.
That's eight atmospheres of boost.....
Depends on how you measure the horsepower too -- aircraft engines are rated on a real dynamometer driving a large generator into a resistance grid. I've seen one, pretty big setup for even the Wasp Majors (9 cylinder, 750 hp or 950 hp) Forestry Canada used for water bombers in Twin Otters. I think rate of acceleration "dynamometers" overestimate hp, and the method of calculating hp by the amount of fuel burned (like drag racers do) isn't worth discussing except to say it's a joke. It's sustained hp too, not a few seconds.
Real horsepower is difficult to measure since it's a measure of work done (which is how I know drag racers hp numbers are silly, it only takes 1500 hp or so to accelerate a 1500 lb vehicle to 300 mph in a quarter mile -- the 2000 hp Wright Twin Cyclone hauled a 40,000 lb Corsair up to 70 mph in 300 ft and required water and methanol injection to do it).
Gasoline engines are limited to compression ratios less than combustion temp (else they detonate) unless you are doing something like direct injection after TDC or something, or using exotic fuels instead of 100 octane gasoline.
My brother lucked into a Ford Super Cobra Jet engine he's rebuilt for my nephew, very rare engine as it was only sold for a year or two in Turino's, of all things. Death trap on wheels, junk car with real 600 hp (or more). You can fit your fist into the intake runners if you have small hands, a 10 oz coke can fill drop right in. Nascar dropped it, I suspect mainly because it ate Chevy 454s for lunch.