Short and sweet:
Guillaume Thierry and Yan Jing Wu ran a very clever test on native Chinese speakers reading in a second language, English, to address this issue:
Some studies in cognitive neuroscience have suggested that fluent bilinguals can effectively inhibit their first language when accessing word meaning in their second language based on the word form (1). However, this finding conflicts with functional neuroimaging data showing overlapping cortical representation of the two languages (2, 3). A number of psycholinguistic experiments have also suggested that the two languages mastered by one individual are constantly coactivated and interactive (4–7), whereas others have provided evidence for language independence (8, 9). It therefore remains an open question whether or not bilingual individuals can effectively suppress all interference from their first language when processing their second language (10).
In short, when you're speaking or reading your second language, are you operating just with that language, or are you constantly referring to your native tongue?
Thierry and Wu gave native Chinese speakers pairs of English words and asked them to say if they were related in meaning or not. Unknown to these speakers, half the time the Chinese translations of the paired words involved a character repetition. In other words, if you sat down and bothered to translate the English into Chinese, half the time you'd be looking at homonyms. Their testing revealed this:
Whereas the hidden factor failed to affect behavioral performance, it significantly modulated brain potentials in the expected direction, establishing that English words were automatically and unconsciously translated into Chinese. Critically, the same modulation was found in Chinese monolinguals reading the same words in Chinese, i.e., when Chinese character repetition was evident.
In other words, even if you don't know you're doing it, yes, you are constantly referring back to your mother tongue when you're operating in a second language.