In this paper, we focus on classifying cardiac arrhythmias. The MIT-BIH database is used with 14 original classes of labeling which is then mapped into 5 more general classes, using the Association for the Advancement of Medical Instrumentation standard. Three types of features were selected with a focus on the time-frequency aspects of ECG signal. After using the Wigner-Ville distribution the time-frequency plane is split into 9 windows considering the frequency bandwidth and time duration of ECG segments and peaks. The summation over these windows are employed as pseudo-energy features in classification. The “subject-oriented” scheme is used in classification, meaning the train and test sets include samples from different subjects. The subject-oriented method avoids the possible overfitting issues and guaranties the authenticity of the classification. The overall sensitivity and positive predictivity of classification is 99.67 and 98.92%, respectively, which shows a significant improvement over previous studies.