Alzheimer's disease (AD) is a complex neurodegenerative disorder which seriously causes the dementia in elderly people and afflicts millions of people worldwide. Drug discovery for Alzheimer's disease therapy has been a hot research area and a big challenge, in which development of acetylcholinesterase (AChE) inhibitors design was the most active and some AChE inhibitors are commercially available for AD medication already. However, practical using of commercial AChE inhibitors showed their limited usefulness and related adverse effects. Thus, it is extremely urgent to find novel AChE inhibitors with higher potency and less adverse effects. Based on the accurate crystallographic studies about AChE, strategies for multi-binding site AChE inhibitors have been formed, followed by design of the multi-target directed ligands. In this review, the structures and binding modes of commercial AChE inhibitors were briefly discussed, together with the development of AChE inhibitor design for AD therapy: from multi-binding site inhibitors to multi-target directed ligands.