Small cells as one of the key technologies to the success of the next generation cellular networks, as they aim at boosting network capacity and saving energy. Massive deployment of small cell will conduct the difficulty to provide grid power supply to all small cell base stations (SCBSs) in a cost effective way. Thus, adjustments and transmission strategies will be needed in powering small cell networks by renewable energy sources. This work focused on minimizing outage probability in small cell powered by energy harvesting (EH) by employing the data-driven stochastic energy harvesting model which established by real data record. We propose outage-aware power allocation (OPA) scheme as an optimal transmission policy for two-way communication system between base station (BS) and user (UE) in one single small cell network where each node is solar-powered and equipped with a finite capacity battery. We extend the previous work in term of minimizing outage probability by adapting user scheduling, spectrum efficiency and transmission power to system factors such as solar Energy state information (ESI), battery energy and channel fading with the use of Markov decision process (MDP).