Longitudinal data often arise in clinical trials when measurements are taken from subjects repeatedly over time so that data from each subject are serially correlated. In this paper, we seek some covariance matrices that make the regression parameter estimates robust to misspecification of the true dependency structure between observations. Moreover, we study how this choice of robust covariance matrices is affected by factors such as the length of the time series and the strength of the serial correlation. We perform simulation studies for data consisting of relatively short (N=3), medium (N=6) and long time series (N=14) respectively. Finally, we give suggestions on the choice of robust covariance matrices under different situations.