Text this: Spectral signatures of the Markovian to non-Markovian transition in open quantum systems