Welcome to the site !!
I guess it depends on your needs ..... The stock X muffler/exhaust is fine if you plan on using the bike for its intent purposes , which is trail riding ...... you can add a exhaust , but if it is not meant for the bike , it is going to cost you money for little gain and a bunch of noise ... noise is a dirt bikes enemy ... the louder the bike , the more complaints , especially if where you ride is near neighbors , or others who do not think dirt bikes are welcome or cool ....
So , to answer your question , I would personally keep it quiet and stock with the stock exhaust and trail ride the bike if that is what you got it for , the improvements you gain from the exchange will be minimal , if I was going to spend money making a bike better , I would spend my money on getting the suspension set up for me .... much more gains with that , than a muffler and rejetting , for a 1-3 HP gain at best , if your intentions are to race the bike , then I would keep the exhaust and get the bike to match the muffler , as opposed to trying to make a trail bike a race bike .... Again , I have no idea what your intended purpose is for the bike , just guessing .......... What I would suggest doing is going to the FMF website , and looking up part numbers (if you have them ) or just looking up the 06 250X and see what the interchange is or fitment is listed for those parts , it could be that it is just the header that needs to be swapped/replaced and your good to go ... or you will find the muffler you have will not bolt up (I am guessing the subframe mount is in the wrong location as well) even with the correct header ...... on some of the bikes only a certain year will fit , for example , on a 06 450X , only a 05 450R muffler will directly bolt up , but a 06 will not , so that could be the issue you are having if they followed suit with the 250X as they did with the 450X ..... Hope that helped .....