Diffusion-based molecular communication has become a promising scheme for communication between nanoscale devices, and various modulation schemes have recently been proposed, including type, quantity, and concentration modulation. In this thesis, we investigate molecular communication by separating it into two categories: coherent molecular communication and non-coherent molecular communication, which are based on the adopted signaling and detection methods. For coherent molecular communication, we study modulations that convey information in molecular quantity or molecular type. Due to the randomness of each molecule in the diffusion channel, problems such as the crossover effect and the inter-symbol interference arise which undermine the system performance. This thesis provides algorithms such as ISI cancellation and threshold-based detection algorithm to deal with the problems. Moreover, it is shown by mathematical derivations and computer simulations that the proposed quantity-type modulation, which is designed against the bad channel effects, has reliable performance. For non-coherent molecular communication, we construct a stochastic model to describe the concentration magnitude sensed by the receiver. The model enables more modulation designs since it is generalized to the case that the transmitter send any continuous wave to the the receiver. It also allows better design for detection algorithm. Amplitude modulation and pulse-position modulation in non-coherent molecular communication are studied and compared by using the proposed expansion-based detection as well as the widely-used sampling-based detection. Through simulation, it is proved that the expansion-based detection outperforms the sampling-based detection.