Background: This study evaluated the inter-rater reliability of nurse to physician using the current national Taiwan triage scale, a 4-level triage system. The scale's validity was also assessed by comparing emergency department (ED) length of stay and hospital admission rates for both pediatric and adult patients based on their triage score. Methods: Triage nurses (TNs) documented their patient assessments in writing on a nursing chart. Emergency physicians (EPs) entered a triage score directly into the computer record for each patient encounter. Triage data from the two sources for the period from January 1, 2001 to December 31, 2003 were used for analysis. A systematic sampling method using a 100:1 ratio was employed to select the study subjects. The extracted chart and computer log data were then compared. Results: Among the 1541 patients sampled, the overall agreement between EPs and TNs was 71.3%, with unweighted and quadratic weighted kappa values of 0.25 and 0.36, respectively. When stratified by age, the unweighted and quadratic weighted kappa values for the pediatric patients were 0.27 and 0.38, and for the adult patients were 0.24 and 0.36, respectively. The odds ratio for admission rate prediction by triage category was more than twice as high for EPs than TNs. Conclusion: The study revealed poor inter-rater reliability between nurses and physicians using the Taiwan triage scale. Future studies need to focus on inter-rater reliability within the same provider groups to determine what improvements are required.