Teracyclin (an antibiotic) or Neosuflex is what he needs. To me it sounds as if your new Beta has fin rot, which is caused by dirty water. Still eating and swimming about are good signs though! Use the Teracyclin until you see fins begin to grow back, then switch to Betta-Max once a month , a vitamin rich suppliment that will aid him in healing and won't hurt him. You can also add a pinch of aquarium or rock salt every time you clean his bown.
Also keep his water as clean as possible, by removing 1/3 of it every 4-5 days, and scrubbing it out when scum begins to build up. This will keep fin-rot from reoccuring!