MSS vs MTU – What is the Difference Between MSS and MTU?

Am I right in saying that an Ethernet frame MTU is 1526 while the MTU at the IP layer is 1500?

The Ethernet MTU is 1500 bytes, meaning the largest IP packet (or some other payload) an Ethernet frame can contain is 1500 bytes. Adding 26 bytes for the Ethernet header results in a maximum frame (not the same as MTU) of 1526 bytes.

Does the MTU change at each phase of encapsulation, or is the term "MTU" only meant to define the maximum size of a packet at layer 3?

The MTU is often considered a property of a network link, and will generally refer to the layer 2 MTU. The limits at layer 3 are far higher (see below) and cause no issues.

The length of an IP packet (layer 3) is limited by the maximum value of the 16 bit Total Length field in the IP header. For IPv4, this results in a maximum payload size of 65515 (= 2^16 - 1 - 20 bytes header). Because IPv6 has a 40 byte header, it allows for payloads up to 65495. And IIRC using the Jumbo Payload header extension, IPv6 could allow packets up to 4 GB...

When setting up a TCP connection, a Maximum Segment Size (MSS) is agreed upon. This could be considered an MTU at layer 4, but it is not fixed. It is often set to the largest payload that can be sent in a TCP segment without causing fragmentation, thus reflecting the lowest layer 2 MTU on the path. With an ethernet MTU of 1500, this MSS would be 1460 after subtracting 20 bytes for the IPv4 and TCP header.

From RFC6691

When calculating the value to put in the TCP MSS option, the MTU
value SHOULD be decreased by only the size of the fixed IP and TCP
headers and SHOULD NOT be decreased to account for any possible IP or TCP options; conversely, the sender MUST reduce the TCP data length
to account for any IP or TCP options that it is including in the
packets that it sends.

(emphasis mine)

So the problem is avoided.

Edit : as pointed out by hertitu the cited RFC is not an Internet Standards Track, so let's check RFC1122 which is an Internet Standard.

The maximum size of a segment that TCP really sends, the "effective send MSS," MUST be the smaller of the send MSS (which reflects the available reassembly buffer size at the remote host) and the largest size permitted by the IP layer:

           Eff.snd.MSS =

              min(SendMSS+20, MMS_S) - TCPhdrsize - IPoptionsize

where:

        *    SendMSS is the MSS value received from the remote host,
             or the default 536 if no MSS option is received.

        *    MMS_S is the maximum size for a transport-layer message
             that TCP may send.

        *    TCPhdrsize is the size of the TCP header; this is
             normally 20, but may be larger if TCP options are to be
             sent.

        *    IPoptionsize is the size of any IP options that TCP
             will pass to the IP layer with the current message.

As you can see the MSS must take into account both the IP option size and the TCP options.

Concerning :

However, TCP has no knowledge of IP options as it is on top of IP layer, then how can it assemble segments correctly?

TCP is responsible for passing the IP options requested by the application to the IP layer, so it can compute what will be the IP option size.