There's a series of changes you need to make in addition to just running the patch.

I've used the 1.9.1.1 patch as an example here, but sadly, the patches differ dependent on release.

Custom template files

If you have modified or overridden the base/default template with your own package/design, then you'll need to manually apply the patches as necessary to the following.

In app/design/frontend/base/default/template/checkout/cart.phtml,

- <button type="button" title="<?php echo $this->__('Continue Shopping') ?>" class="button btn-continue" onclick="setLocation('<?php echo $this->getContinueShoppingUrl() ?>')"><span><span><?php echo $this->__('Continue Shopping') ?></span></span></button>
+ <button type="button" title="<?php echo Mage::helper('core')->quoteEscape($this->__('Continue Shopping')) ?>" class="button btn-continue" onclick="setLocation('<?php echo Mage::helper('core')->quoteEscape($this->getContinueShoppingUrl()) ?>')"><span><span><?php echo $this->__('Continue Shopping') ?></span></span></button>

In app/design/frontend/base/default/template/checkout/cart/noItems.phtml,

- <p><?php echo $this->__('Click <a href="%s">here</a> to continue shopping.', $this->getContinueShoppingUrl()) ?></p>
+ <p><?php echo $this->__('Click <a href="%s">here</a> to continue shopping.', Mage::helper('core')->quoteEscape($this->getContinueShoppingUrl())) ?></p>

In app/design/frontend/base/default/template/checkout/onepage/failure.phtml

-<p><?php echo $this->__('Click <a href="%s">here</a> to continue shopping.', $this->getContinueShoppingUrl()) ?></p>
+<p><?php echo $this->__('Click <a href="%s">here</a> to continue shopping.', Mage::helper('core')->quoteEscape($this->getContinueShoppingUrl())) ?></p>

In app/design/frontend/base/default/template/rss/order/details.phtml,

- <?php echo $this->__('Customer Name: %s', $_order->getCustomerFirstname()?$_order->getCustomerName():$_order->getBillingAddress()->getName()) ?><br />
- <?php echo $this->__('Purchased From: %s', $_order->getStore()->getGroup()->getName()) ?><br />
+ <?php $customerName = $_order->getCustomerFirstname() ? $_order->getCustomerName() : $_order->getBillingAddress()->getName(); ?>
+ <?php echo $this->__('Customer Name: %s', Mage::helper('core')->escapeHtml($customerName)) ?><br />
+ <?php echo $this->__('Purchased From: %s', Mage::helper('core')->escapeHtml($_order->getStore()->getGroup()->getName())) ?><br />

In app/design/frontend/base/default/template/wishlist/email/rss.phtml,

- <?php echo $this->__("RSS link to %s's wishlist",$this->helper('wishlist')->getCustomerName()) ?>
+ <?php echo $this->__("RSS link to %s's wishlist", Mage::helper('core')->escapeHtml($this->helper('wishlist')->getCustomerName())) ?>

In app/design/frontend/default/modern/template/checkout/cart.phtml,

- <button type="button" title="<?php echo $this->__('Continue Shopping') ?>" class="button btn-continue" onclick="setLocation('<?php echo $this->getContinueShoppingUrl() ?>')"><span><span><?php echo $this->__('Continue Shopping') ?></span></span></button>
+ <button type="button" title="<?php echo Mage::helper('core')->quoteEscape($this->__('Continue Shopping')) ?>" class="button btn-continue" onclick="setLocation('<?php echo Mage::helper('core')->quoteEscape($this->getContinueShoppingUrl()) ?>')"><span><span><?php echo $this->__('Continue Shopping') ?></span></span></button>

.htaccess rules

If you aren't using a .htaccess compatbile web server, then you'll need to manually add the following deny rules.

For Nginx,

location /downloader/Maged/ { deny all; }
location /downloader/lib/   { deny all; }

File permissions

If the user of your web server PHP process isn't the owner of the document root and relies on group permissions, and the var/report or var/log directories are removed - you will encounter issues with the new default file permissions.

Eg. In the following scenario,

PHP User:        www-data
Doc Root User:   sonassi
Doc Root Group:  www-data
www-data members: sonassi, www-data

The revised default file permissions of 0750, will strip write permissions from the group - which will prohibit your web server from being able to write to the directory.

Equally, if you rely on the everyone permission, all access will be stripped.