Overriding component parameters when using the Castle.MicorKernel turned out to be kinda weird. Anyway, here's one way to do it.

IoC.Container.Kernel.RemoveComponent("ComponentA");
IoC.Container.Kernel.AddComponentInstance(
    "ComponentA", 
    typeof (ComponentA),
    new ComponentA("Customer parameter for testing."));

I needed to override the parameter value of ComponentA for some unit tests. The parameter is normally set in the configuration file. After trying a few different approaches, I ended up with the code above. It's more of an end to end unit test, and as such ComponentA will be resolved as a result of another Component being resolved.