Hello,
The output voltage from STWLC38 is not supposed to oscillate, it should be stable as it is a LDO output.
Could you attach a zoom of the period of time where VIN2=5V and then it falls ? It is difficult to see on the plot the behavior of VIN2 and if it exceeds 5.5V.
If you suspect a overvoltage protection by TPS2116, you could try (for testing purpose) to set Vout=4.5V using STWLC38 GUI.
Then, which configuration do you use inside STWLC38 ? I understood that you have modified several parameters, it may be a reason of instability. For instance, if you have increased the minimum operating frequency, you will get less power available at Vout. In your 3rd case, have you tested that Vout is still stable when you draw some current ?
What is the maximum of power that you need at Vout ?
Regards,
Didier