Electronic – How to achieve low power consumption with ATTINY85

I am creating a project that uses an ATTINY85. Most of the time, the circuit should be off and consume as little power as possible. According to the data sheet, in power-down mode, the microcontroller consumes 0.1 μA at 1.8V.
I am powering the circuit using two AAA batteries in series, which gives around 3V, so I expect an higher power consumption, but still on the same order, but so far I am getting 300 μA in power-down mode.

I have removed everything from my code except for the power management code to try to isolate the problem:

#include "Arduino.h"
#include <avr/sleep.h>
#include <avr/power.h>

void setup()
{
  cli();  // Disable interrupts

  // Reduce the clock frequency to conserve power
  clock_prescale_set(clock_div_128);

  // Disable all modules
  power_all_disable();
  set_sleep_mode(SLEEP_MODE_PWR_DOWN);

  // Configure all ports as output
  // (does not seem to make any difference)
  DDRB = 0b0011111;
  PORTB = 0;
}    

void loop()
{
  sleep_enable();
  sleep_bod_disable();
  sleep_cpu();
}

As can be inferred from the code, I am using the Arduino library to program because it was quicker to setup, but I do no think that this would make any difference. Am I wrong?

The fuses are E:FF, H:DF, L:62, so I am using the internal 8MHz oscillator, divided by 8. I further use the clock prescaler to divide it by 128, because it seems that it should reduce the power consumption.

My circuit is as follows:

This diagram does not show it, but the resistors are connected the anodes of four LEDS, whose cathodes are connected to ground (so the leds turn on when the pins are high).

I am measuring the current with a multimeter whose lowest scale setting is 200 μA, so I believe that I should be able to measure this current with reasonable accuracy.

What am I missing here? Why is my current consumption higher than expected?