This paper presents the flight control design of a rotorcraft-based unmanned aerial system (RUAS) using an improved PID tuning method based on pole placement (PP) and Linear Quadratic Regulator (LQR) approaches to achieve the desired time response specification. The rotorcraft-based unmanned aerial system (RUAS) flight poses a challenging control problem due to the unstable nonlinear system with fast responsive dynamics. This work utilized X-Plane flight simulator to test the control gain values of the autopilot system, which is designed using a PID controller in the LabVIEW environment using the Software in the Loop (SIL) approach. Several simulated flight tests were conducted to validate the proposed controllers' performance and evaluate the effectiveness of the control systems. The results indicate that PID controllers tuned using pole placement (PP) and linear quadratic regulator (LQR) methods can produce satisfactory position tracking performance in hovering maneuvers within ±0.4 m and ±0.1 m for x and y-positions, respectively. The LQR tuning method will produce better trajectory tracking performance with less oscillation compared to the PP method and showed adequate robustness in dealing with 4.5 m/s wind turbulence in a circular path tracking simulation with 2.701° heading error accuracy.