A good charger for you battery should charge at between 0.5 and 1 amp/hour. It works out so that you're probably going to need to leave it on the charger for at least 10 hours to get a full charge into it.
I'd do that before I tried replacing the battery. If you want too you can check the voltage across the battery terminals, it should be above 12.8volts at rest after it's finished charging and has sat for about 30 minutes. If you can't hit that then you know you need a new battery.
Of course I'm generalizing from a single example here, but everyone does that. At least I do.