Louis, 62, slammed 31-year-old Cheryl during a recent interview, labelling her 'boring' and 'irrelevant'.
Cheryl hit back on Twitter, but Simon thinks they're arguments are wasted on social media.
Appearing on This Morning, Simon said: "I wish they'd fight more on there show actually. Why do it on Twitter?"
And, hinting again that Louis won't be back this year, Simon added: "What we should do, is just invite Louis down every week just to argue with Cheryl!"
Following his harsh comments, Cheryl tweeted: "I'm beginning to wonder if Louis has some kind of obsession with me…
"I mean this is every year whether I am or am not a part of the show. And btw… this is the same man asking me what he should say to the contestants on our way to the stage bc 'he doesn't know' [sic]."
While Louis' future on the show remains uncertain, it is thought that Cheryl will be returning when The X Factor hits our screens again later this year.
Radio 1 DJ Nick Grimshaw has been hotly tipped to replace Louis as a judge.