There are two ways to add tabs to the product edit page. Tabs are directly associated to attribute set groups.

To add from the admin interface:

1. Go to Admin > Stores > Attributes > Attribute Set.
2. Select your attribute set (default, or whatever the case).
3. Under the center column 'Groups', click 'Add New'.
4. Enter your group name, then add attributes to it.

This will add a tab corresponding to the group.

To do it by code:

Create a module, if you don't already have one.

Create a class {module}\Setup\InstallData:

<?php

namespace My\Module\Setup;

use Magento\Framework\Setup\ModuleContextInterface;
use Magento\Framework\Setup\ModuleDataSetupInterface;

/**
 * Install attributes
 */
class InstallData implements \Magento\Framework\Setup\InstallDataInterface
{
    /**
     * @var \Magento\Catalog\Setup\CategorySetupFactory
     */
    protected $categorySetupFactory;

    /**
     * Init
     *
     * @param \Magento\Catalog\Setup\CategorySetupFactory $categorySetupFactory
     */
    public function __construct(
        \Magento\Catalog\Setup\CategorySetupFactory $categorySetupFactory
    ) {
        $this->categorySetupFactory = $categorySetupFactory;
    }

    /**
     * Installs data for a module
     *
     * @param ModuleDataSetupInterface $setup
     * @param ModuleContextInterface $context
     * @return void
     */
    public function install(ModuleDataSetupInterface $setup, ModuleContextInterface $context)
    {
        /** @var \Magento\Catalog\Setup\CategorySetup $categorySetup */
        $categorySetup = $this->categorySetupFactory->create(['setup' => $setup]);

        $setup->startSetup();

        $categorySetup->addAttribute(
            \Magento\Catalog\Model\Product::ENTITY,
            'custom_attribute',
            [
                'type'                  => 'int',
                'label'                 => 'Custom Attribute',
                'input'                 => 'select',
                'source'                => 'Magento\Eav\Model\Entity\Attribute\Source\Boolean',
                'sort_order'            => 100,
                'global'                => \Magento\Eav\Model\Entity\Attribute\ScopedAttributeInterface::SCOPE_STORE,
                'group'                 => 'My Custom Tab', // If this does not exist, a new group will be created.
                'is_used_in_grid'       => true,
                'is_visible_in_grid'    => true,
                'is_filterable_in_grid' => true,
                'used_for_promo_rules'  => true,
                'required'              => false,
            ]
        );

        $setup->endSetup();
    }
}

This will add a yes/no product attribute, and assign it to attribute group 'My Custom Tab'. The label can be anything.

If you want a custom for your form field(s), you could define your own input renderer for the attribute.

You can also add a group explicitly, and place it within the 'advanced' section, like so:

// Add new tab
$entityTypeId   = $categorySetup->getEntityTypeId(\Magento\Catalog\Model\Product::ENTITY);
$attributeSetId = $categorySetup->getAttributeSetId($entityTypeId, 'Default');

$categorySetup->addAttributeGroup($entityTypeId, $attributeSetId, 'My Custom Tab', 65);
$categorySetup->updateAttributeGroup(
    $entityTypeId,
    $attributeSetId,
    'My Custom Tab',
    'attribute_group_code',
    'my_custom_tab'
);
$categorySetup->updateAttributeGroup(
    $entityTypeId,
    $attributeSetId,
    'My Custom Tab',
    'tab_group_code',
    'advanced'
);