It remembers the MAC of the computer attached to that port.
if you have four computers connected to a switch:
PC1 with MAC1 connected to PORT1
PC2 with MAC2 connected to PORT2
PC3 with MAC3 connected to PORT3
PC4 with MAC4 connected to PORT4
Imagine you just turned the switch on, and the MAC table is empty.
When you send a packet from PC1 to PC2, the switch doesnt yet know where the PC with MAC2 is (on which port), but it knows that it recieved a packet from MAC1 on PORT1 (and it adds this entry to the MAC table). Since it doesnt know where PC2 is, it sends the packet to all ports (except PORT1, since the packet came from there.
All three other PCs get the packet, PC3 and PC4 ignore it, but PC2 probably replies. Since the switch just recieved a packet from a PC with MAC2 on PORT2, it adds this entry to the MAC table. Since it also has an entry for MAC1 on PORT1, it sends the packet there (and only there).
After such a packet exchange (eg. ping), the state of the MAC table would be:
a device with MAC1 is located on PORT1
a device with MAC2 is located on PORT2
The mechanism is called Address Resolution Protocol (ARP). Every ethernet IPv4 device ARPs to resolve ethernet mac addresses for target IPs. IP to mac mappings are stored in each device's ARP table (the phone book in your analogy).
To simplify: In most cases, to resolve the MAC address associated with an IP address, you send a broadcast ARP packet (to all devices in the network), asking who has that IP address. The device with that IP address replies to the ARP (with its MAC address).
Best Answer
MAC addresses only need to be unique in a local broadcast domain, not globally, so re-use of MAC addresses in different networks usually isn't a problem.
The internet isn't one global broadcast domain and thus needs to be divided into many blocks of addresses assigned to different ISP's and each ISP divides his blocks into smaller blocks for different customers/services. To allow each of these smaller blocks to contain many MAC addresses you need to have the IP-space much bigger than the MAC address space.