Bilingualism and its cognitive impacts have drawn increasing interest. Recently, inconsistencies in the findings have raised discussions on what might have caused such discrepancies and how evidence should be evaluated. This review tries to shed new light onto the reasons for the inconsistencies by taking a novel perspective. Motivated by the finding that bilingualism affects response time distribution profiles, particularly findings that suggest bilinguals have fewer long responses, we investigated the relation between maximum response times allowed/included in the analysis of an experiment and the finding of a bilingual advantage. We reviewed 68 experiments from 33 articles that compared monolingual and bilingual speakers’ performance in three commonly used non-verbal interference tasks (Simon, Spatial Stroop and Flanker). We found that studies that included longer responses in their analysis were more likely to report a bilingualism effect. We conclude that seemingly insignificant details such as the data trimming procedure can have a potential impact on whether an effect is observed. We also discuss the implication of our findings and suggest the usefulness of more fine-grid analytical procedures.