Oxi is tricky. Some of the side effects actually worsen in the weeks after your last dose!
Neuropathy, especially, tends to keep building for a few more weeks, then levels out and starts going down. The other side-effects -- nausea, diarrhea, mouth sores, low blood counts, fatigue, loss of taste buds/appetite, muscle cramps -- they start going away almost immediately unless you are on other chemo.
For me, personally, there was a huge difference in my muscle cramping, fatigue, and loss of taste buds when I stopped oxi. I continued to take Avastin and Xeloda (pill form of 5FU) so I still had side effects from them, but oxi was responsible for my worst symptoms and I felt so much better when it was gone!
If you're fine with the pump, stick with it. Xeloda has its own set of side-effects but to me they were manageable and I didn't want to haul the pump around. If you're managing the pump's side-effects pretty well, then I'd stay with it if I were you.