With the deployment of more and more edge devices and the improvement of intelligent technologies level, research on the cognitive Internet of Things for fog-based smart home emerges as the times require. In this context, the study of abnormal network traffic also encountered new problems, such as data islands and user privacy leakage. In order to solve these problems, we propose an anomaly detection method based on federated learning and homomorphic encryption on cognitive Internet of Things for fog-based smart home. This model is a new data sharing mode, which only needs a kind of processing of shared data, not the shared data itself. Firstly, a model based on federated learning on cognitive Internet of Things for fog-based smart home is proposed to solve the problem of data island through multi model cooperation of different terminals. Secondly, to protect the user privacy, we use homomorphic encryption to achieve encrypted transmission. By using homomorphic encryption, data aggregation of model parameters is realized. After analysis, the model can be extended to multi-dimensional or multi-level. Finally, security analysis, model training performance, computational complexity and communication cost are investigated. The simulation results show that the proposed scheme behave well in both cost and performance.