Absolutely test the alternator. It sounds like a classic alternator symptom.